Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X....................................X.X.X.............XX...] duration: 434.417896ms FAIL (2025-08-19 17:38:12 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 434.438765ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X...X.................] duration: 404.224575ms FAIL (2025-08-19 17:38:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 404.241175ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.....................X...........X.....................X....] duration: 185.122071ms FAIL (2025-08-19 17:38:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 185.144633ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X...X................] duration: 609.026921ms FAIL (2025-08-19 17:37:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 609.078538ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X....................................X.X.X.............XX..] duration: 1.785086974s FAIL (2025-08-19 17:37:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 1.785112933s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X....X................] duration: 431.160013ms FAIL (2025-08-19 17:37:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 431.229051ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 543.42018ms FAIL (2025-08-19 17:37:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 543.444444ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 1.413940834s FAIL (2025-08-19 17:37:30 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.413963405s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [......................X...........X.....................X...] duration: 348.430948ms FAIL (2025-08-19 17:37:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 348.455855ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 1.357557961s FAIL (2025-08-19 17:36:54 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.357601112s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X...X...............] duration: 471.952216ms FAIL (2025-08-19 17:36:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 471.976962ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.......................X...........X.....................X..] duration: 223.031961ms FAIL (2025-08-19 17:36:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 223.052098ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X....X...............] duration: 392.581528ms FAIL (2025-08-19 17:36:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 392.60425ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 458.781008ms FAIL (2025-08-19 17:36:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 458.795385ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X....................................X.X.X.............XX.] duration: 426.241649ms FAIL (2025-08-19 17:36:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 426.260384ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X....................................X.X.X.............XX] duration: 1m0.001030524s FAIL (2025-08-19 17:36:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 1m0.001041675s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [........................X...........X.....................X.] duration: 202.785322ms FAIL (2025-08-19 17:35:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 202.80546ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 1.561541704s FAIL (2025-08-19 17:35:50 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.56155488s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X...X..............] duration: 368.49804ms FAIL (2025-08-19 17:35:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 368.516416ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X....X..............] duration: 409.73962ms FAIL (2025-08-19 17:35:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 409.764206ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [.........................X...........X.....................X] duration: 1m0.001263989s FAIL (2025-08-19 17:35:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 1m0.001283697s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 496.745741ms FAIL (2025-08-19 17:35:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 496.770336ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 1.725840112s FAIL (2025-08-19 17:35:26 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.725867054s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X....................................X.X.X.............X] duration: 1m0.001256671s FAIL (2025-08-19 17:35:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:24.516401523 (264h8m5.247669153s) [upload=274694]: Starting upload Aug 19 17:34:24.516444794 (264h8m5.247712424s) [upload=274694]: Done starting upload Aug 19 17:34:24.516495879 (264h8m5.247763509s) [upload=274694]: Uploading segments... Aug 19 17:34:24.516678072 (264h8m5.247945703s) [upload=274694]: Got next segment. Inline: false Aug 19 17:34:24.516718889 (264h8m5.247986519s) [upload=274694]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:24.516722566 (264h8m5.247990186s) [upload=274694]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:34:24.540141574 (264h8m5.271409194s) [seg_pos={0 0}, upload=274694]: Begin upload segment... Aug 19 17:34:24.540182752 (264h8m5.271450372s) [seg_pos={0 0}, upload=274694]: Done begin upload segment. Aug 19 17:34:24.540194484 (264h8m5.271462114s) [noise=true , address=45.91.116.10:4101 , piece_id=EE3UZUBTOZVHUY267DNSODSD3WOTZJGVWWBQ645MV2NE7NIM7E4A, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.540211827 (264h8m5.271479447s) [noise=true , address=104.36.82.11:4137 , piece_id=QQ42SWXFESMBC3UQHCILUMZWEF6DPMYYRRIXUX2KV4DRSA6HNBQQ, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.540668955 (264h8m5.271936586s) [noise=true , address=77.90.31.13:4646 , piece_id=OC2UXXYDEQNWXWA6QM2HU3F72HK45YBQMYD5AFIOWH7KI7KUN6SQ, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.540731734 (264h8m5.271999364s) [noise=true , address=77.90.31.15:4715 , piece_id=XPOPYM3LXTLAHYE2ININGP2M73BR5EQPYR3IFI3GJV3UFFQTPK3Q, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.541044472 (264h8m5.272312102s) [noise=true , address=104.36.82.18:4396 , piece_id=ILOSUOP3FDL4YHD7LHP7KF6LEXSGD4EBJKI2KDTQOLN3MCIM6U6Q, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.541222315 (264h8m5.272489945s) [noise=true , address=45.91.116.7:4015 , piece_id=RXTK2ZPDFK6YWMICEWXLVXCTQ62ZLBFZFK34OIEUD7AX6URQ3V3Q, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.541395210 (264h8m5.27266284s) [noise=true , address=45.91.116.7:4043 , piece_id=PQ3DNM7Q6IYW7V6GIOLGN45R6RGLWOG6UCZMCEA2Z7RORFZW7P3Q, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.541689273 (264h8m5.272956903s) [noise=true , address=79.127.205.246:4269 , piece_id=KZJEJILJ3KE6JN7T3YHUYJ7J52PQDGP2SQUO3AM2O4TJ36UXM6DQ, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.541745228 (264h8m5.273012858s) [noise=true , address=77.90.31.21:4904 , piece_id=IBUG44EXA2VKL3F5HO54ZAM5VSZBAS5AGWOBMDTXRTQQFN6CGMJQ, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.542129120 (264h8m5.27339675s) [noise=true , address=104.36.82.23:4472 , piece_id=CW7ZF2T5KRXJ4JBFAGES2C3IVCAD6DYKOW4R5JMQSGB35GEYZF2Q, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.542191176 (264h8m5.273458806s) [noise=true , address=104.36.82.17:4362 , piece_id=NBSMTYWGTTPLMLN5S3SJS6BEV5WCNC2Z2RSSV7I6GANKFVJGM5PA, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.542485749 (264h8m5.27375338s) [noise=true , address=104.36.82.60:4365 , piece_id=P5PYJW2EWTIRLW6VQJ54MBYPB32NB45PVR5JKUCGQLOBUT6FNHMQ, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.542666189 (264h8m5.273933819s) [noise=true , address=104.36.82.5:4852 , piece_id=3YAUB35D7JJZHYGILHBYKAH7Y66SODWNIYZRR7RSVMZ3RP23EWIQ, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.542887956 (264h8m5.274155586s) [noise=true , address=45.91.116.6:4983 , piece_id=YB6AMQP6XNCOU6G5YP3TNR4WQF5M7M35NRLWX76BFJHRHOGHEEKA, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.543243624 (264h8m5.274511254s) [noise=true , address=104.36.82.7:4916 , piece_id=A57FC6SUX4LMWOODYD66TOYGPF7SK2JJ6U3MBA4CWTYW6NPKGSYQ, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.544262819 (264h8m5.275530449s) [noise=true , address=104.36.82.44:4869 , piece_id=IJBXRUS2LD23ODDKZCORFMNQLYOJTM4DVRWJHDRE34ZQFU5HZQ5Q, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.545286202 (264h8m5.276553832s) [noise=true , address=77.90.31.11:4580 , piece_id=P5WCOAAPCMPGZRCRK4CSZMIHKKUKF4JEIQQGJZXXLK5NFIOVFYDA, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.545671496 (264h8m5.276939126s) [noise=true , address=77.90.31.18:4809 , piece_id=ENCHU2CHOPNZ5LCSDX6YQZRVVBDTG7RLNL3DVRFUGVBOIWAKDIWA, seg_pos={0 0}, upload=274694]: Uploading piece... Aug 19 17:34:24.657636301 (264h8m5.388903921s) [upload=274694]: Next returned nil segment Aug 19 17:34:24.657646440 (264h8m5.38891405s) [upload=274694]: Waiting for error group managing segments... Aug 19 17:34:24.666835538 (264h8m5.398103158s) [noise=true , address=45.91.116.10:4101 , piece_id=EE3UZUBTOZVHUY267DNSODSD3WOTZJGVWWBQ645MV2NE7NIM7E4A, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.667614420 (264h8m5.398882041s) [noise=true , address=45.91.116.6:4983 , piece_id=YB6AMQP6XNCOU6G5YP3TNR4WQF5M7M35NRLWX76BFJHRHOGHEEKA, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.682924742 (264h8m5.414192372s) [noise=true , address=104.36.82.44:4869 , piece_id=IJBXRUS2LD23ODDKZCORFMNQLYOJTM4DVRWJHDRE34ZQFU5HZQ5Q, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.683731679 (264h8m5.414999299s) [noise=true , address=77.90.31.21:4904 , piece_id=IBUG44EXA2VKL3F5HO54ZAM5VSZBAS5AGWOBMDTXRTQQFN6CGMJQ, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.695362786 (264h8m5.426630416s) [noise=true , address=45.91.116.7:4015 , piece_id=RXTK2ZPDFK6YWMICEWXLVXCTQ62ZLBFZFK34OIEUD7AX6URQ3V3Q, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.697148903 (264h8m5.428416533s) [noise=true , address=45.91.116.7:4043 , piece_id=PQ3DNM7Q6IYW7V6GIOLGN45R6RGLWOG6UCZMCEA2Z7RORFZW7P3Q, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.698968321 (264h8m5.430235942s) [noise=true , address=104.36.82.23:4472 , piece_id=CW7ZF2T5KRXJ4JBFAGES2C3IVCAD6DYKOW4R5JMQSGB35GEYZF2Q, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.699337706 (264h8m5.430605326s) [noise=true , address=104.36.82.11:4137 , piece_id=QQ42SWXFESMBC3UQHCILUMZWEF6DPMYYRRIXUX2KV4DRSA6HNBQQ, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.701041738 (264h8m5.432309358s) [noise=true , address=104.36.82.18:4396 , piece_id=ILOSUOP3FDL4YHD7LHP7KF6LEXSGD4EBJKI2KDTQOLN3MCIM6U6Q, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.703077613 (264h8m5.434345243s) [noise=true , address=104.36.82.17:4362 , piece_id=NBSMTYWGTTPLMLN5S3SJS6BEV5WCNC2Z2RSSV7I6GANKFVJGM5PA, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.703626706 (264h8m5.434894326s) [noise=true , address=104.36.82.7:4916 , piece_id=A57FC6SUX4LMWOODYD66TOYGPF7SK2JJ6U3MBA4CWTYW6NPKGSYQ, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.704566862 (264h8m5.435834492s) [noise=true , address=104.36.82.60:4365 , piece_id=P5PYJW2EWTIRLW6VQJ54MBYPB32NB45PVR5JKUCGQLOBUT6FNHMQ, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.707566710 (264h8m5.438834329s) [noise=true , address=77.90.31.15:4715 , piece_id=XPOPYM3LXTLAHYE2ININGP2M73BR5EQPYR3IFI3GJV3UFFQTPK3Q, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.713211407 (264h8m5.444479027s) [noise=true , address=77.90.31.13:4646 , piece_id=OC2UXXYDEQNWXWA6QM2HU3F72HK45YBQMYD5AFIOWH7KI7KUN6SQ, seg_pos={0 0}, upload=274694]: Done uploading piece. err: <nil> Aug 19 17:34:24.713232747 (264h8m5.444500377s) [seg_pos={0 0}, upload=274694]: Segment reached optimal threshold of 14 pieces. Aug 19 17:34:24.713380946 (264h8m5.444648566s) [noise=true , address=77.90.31.18:4809 , piece_id=ENCHU2CHOPNZ5LCSDX6YQZRVVBDTG7RLNL3DVRFUGVBOIWAKDIWA, seg_pos={0 0}, upload=274694]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZkWnew57Zj3H7XoUZHPo57Rzfw5awa7DYwuc7chcD4bJ8aGPz): protocol: expected piece hash; context canceled; EOF Aug 19 17:34:24.713389241 (264h8m5.444656862s) [noise=true , address=104.36.82.5:4852 , piece_id=3YAUB35D7JJZHYGILHBYKAH7Y66SODWNIYZRR7RSVMZ3RP23EWIQ, seg_pos={0 0}, upload=274694]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sQvAZS92R7TTJUFjMRieYT6BP4vp2UAan4PHrhPt6m3eHPukN): protocol: write tcp 5.161.89.252:35642->104.36.82.5:4852: use of closed network connection; write tcp 5.161.89.252:35642->104.36.82.5:4852: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:35642->104.36.82.5:4852: use of closed network connection Aug 19 17:34:24.713466626 (264h8m5.444734247s) [noise=true , address=77.90.31.11:4580 , piece_id=P5WCOAAPCMPGZRCRK4CSZMIHKKUKF4JEIQQGJZXXLK5NFIOVFYDA, seg_pos={0 0}, upload=274694]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12HCCmBhKtqUembEH7kRqTjViQ8Kj6WLGtmcwrtcWh1HUVrXB4D): protocol: expected piece hash; context canceled; EOF Aug 19 17:34:24.713509266 (264h8m5.444776896s) [noise=true , address=79.127.205.246:4269 , piece_id=KZJEJILJ3KE6JN7T3YHUYJ7J52PQDGP2SQUO3AM2O4TJ36UXM6DQ, seg_pos={0 0}, upload=274694]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6568Hvk8w9HgV7uD3kqCsK5y1ab6utJohhFK6RDt1rRggfDY): protocol: read tcp 5.161.89.252:57528->79.127.205.246:4269: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:34:24.713526118 (264h8m5.444793749s) [seg_pos={0 0}, upload=274694]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:34:24.713573397 (264h8m5.444841027s) [upload=274694]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:34:24.713576693 (264h8m5.444844314s) [upload=274694]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:24.756023890 (264h8m5.48729152s) [upload=274694]: Done uploading segments... Aug 19 17:34:24.756037877 (264h8m5.487305507s) [upload=274694]: Upload finished. err: <nil> Duration: 1m0.00126703s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 627.752725ms FAIL (2025-08-19 17:34:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 627.779707ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X....X.............] duration: 572.080648ms FAIL (2025-08-19 17:34:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 572.10322ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X...X.............] duration: 483.772206ms FAIL (2025-08-19 17:34:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 483.794588ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 523.464199ms FAIL (2025-08-19 17:34:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 523.489916ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.649867716s FAIL (2025-08-19 17:34:30 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.649885499s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X....................................X.X.X.............] duration: 1m0.001261121s FAIL (2025-08-19 17:34:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 19 17:24:12.613365846 (263h57m53.344633476s) [upload=274521]: Starting upload Aug 19 17:24:12.613406413 (263h57m53.344674043s) [upload=274521]: Done starting upload Aug 19 17:24:12.613493606 (263h57m53.344761226s) [upload=274521]: Uploading segments... Aug 19 17:24:12.613734899 (263h57m53.345002529s) [upload=274521]: Got next segment. Inline: false Aug 19 17:24:12.613764896 (263h57m53.345032526s) [upload=274521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:24:12.613769284 (263h57m53.345036914s) [upload=274521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:24:12.651951109 (263h57m53.383218739s) [seg_pos={0 0}, upload=274521]: Begin upload segment... Aug 19 17:24:12.651990573 (263h57m53.383258193s) [seg_pos={0 0}, upload=274521]: Done begin upload segment. Aug 19 17:24:12.652011803 (263h57m53.383279433s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652035357 (263h57m53.383302987s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652327476 (263h57m53.383595106s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652383030 (263h57m53.38365066s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652565463 (263h57m53.383833093s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652676883 (263h57m53.383944513s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652812508 (263h57m53.384080138s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652949715 (263h57m53.384217345s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653091241 (263h57m53.384358871s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653251271 (263h57m53.384518892s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653305264 (263h57m53.384572884s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653509577 (263h57m53.384777207s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653518354 (263h57m53.384785984s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653743006 (263h57m53.385010626s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653788721 (263h57m53.385056351s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653985512 (263h57m53.385253132s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654086841 (263h57m53.385354471s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654373080 (263h57m53.3856407s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.735474331 (263h57m53.466741961s) [upload=274521]: Next returned nil segment Aug 19 17:24:12.735486304 (263h57m53.466753934s) [upload=274521]: Waiting for error group managing segments... Aug 19 17:24:12.756883830 (263h57m53.48815145s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.766603935 (263h57m53.497871585s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.809581250 (263h57m53.540859941s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.908061427 (263h57m53.639329057s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.147463951 (263h57m53.878731601s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.364439955 (263h57m54.095707575s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:51.706547376 (263h58m32.437815006s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:52.370283683 (263h58m33.101551303s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.816229069 (263h58m36.547496699s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.944464809 (263h58m36.675732439s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.083334942 (263h58m48.814602572s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.405488480 (263h58m49.136756101s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.160489383 (263h58m52.891757013s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.595037822 (263h58m53.326305452s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:12tQRqowH4aDr9h3s1WLs1o7na1ueiafwgUeeHfAB6g7mcE6e1t, address:45.91.116.14:4213): protocol: rpc: dial tcp 45.91.116.14:4213: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595089570 (263h58m53.326357189s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1KvoMM3ptQXSxPhXFe3UoS145h8n3DMT9qmeaoR3bqNa8KLMm9, address:45.91.116.12:4250): protocol: write tcp 5.161.89.252:56044->45.91.116.12:4250: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595102303 (263h58m53.326369933s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1yLCMLdMxDT9HviM2QRVvP1hkektw7WmGxm8QVpp7h7ZVexkjy, address:45.91.116.4:4950): protocol: rpc: dial tcp 45.91.116.4:4950: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595112262 (263h58m53.326379892s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1pX3YNcCwaUeMstL4NNwvnKQmHbmdxLTUc8sDHwyvZGZtzmYgW, address:77.90.31.11:4574): protocol: write tcp 5.161.89.252:9456->77.90.31.11:4574: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595145234 (263h58m53.326412864s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1gnJixpV2uZqEMTxi22y1EAzKCHXaQeBWakhT6AcEkrENWT8CT, address:45.91.116.13:4182): protocol: write tcp 5.161.89.252:9502->45.91.116.13:4182: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595163289 (263h58m53.326430909s) [seg_pos={0 0}, upload=274521]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 19 17:25:12.636371896 (263h58m53.367639527s) [upload=274521]: Done uploading segments... Aug 19 17:25:12.636391424 (263h58m53.367659054s) [upload=274521]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.001278683s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X...........X.....................] duration: 175.269787ms FAIL (2025-08-19 17:34:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 175.293301ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 910.078341ms FAIL (2025-08-19 17:34:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 910.102225ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X...X............] duration: 347.531958ms FAIL (2025-08-19 17:33:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 347.553177ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X....X............] duration: 382.549043ms FAIL (2025-08-19 17:33:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 382.605328ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 1.455026292s FAIL (2025-08-19 17:33:24 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.455044566s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X...X...........] duration: 513.108859ms FAIL (2025-08-19 17:33:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 513.127984ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 378.129751ms FAIL (2025-08-19 17:33:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 378.206545ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................X...........X....................] duration: 195.851303ms FAIL (2025-08-19 17:33:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 195.942505ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X....X...........] duration: 373.622718ms FAIL (2025-08-19 17:32:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 373.644367ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 1.607423326s FAIL (2025-08-19 17:32:31 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.607439426s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X....................................X.X.X............] duration: 551.670819ms FAIL (2025-08-19 17:32:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 19 17:24:12.613365846 (263h57m53.344633476s) [upload=274521]: Starting upload Aug 19 17:24:12.613406413 (263h57m53.344674043s) [upload=274521]: Done starting upload Aug 19 17:24:12.613493606 (263h57m53.344761226s) [upload=274521]: Uploading segments... Aug 19 17:24:12.613734899 (263h57m53.345002529s) [upload=274521]: Got next segment. Inline: false Aug 19 17:24:12.613764896 (263h57m53.345032526s) [upload=274521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:24:12.613769284 (263h57m53.345036914s) [upload=274521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:24:12.651951109 (263h57m53.383218739s) [seg_pos={0 0}, upload=274521]: Begin upload segment... Aug 19 17:24:12.651990573 (263h57m53.383258193s) [seg_pos={0 0}, upload=274521]: Done begin upload segment. Aug 19 17:24:12.652011803 (263h57m53.383279433s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652035357 (263h57m53.383302987s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652327476 (263h57m53.383595106s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652383030 (263h57m53.38365066s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652565463 (263h57m53.383833093s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652676883 (263h57m53.383944513s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652812508 (263h57m53.384080138s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652949715 (263h57m53.384217345s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653091241 (263h57m53.384358871s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653251271 (263h57m53.384518892s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653305264 (263h57m53.384572884s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653509577 (263h57m53.384777207s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653518354 (263h57m53.384785984s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653743006 (263h57m53.385010626s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653788721 (263h57m53.385056351s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653985512 (263h57m53.385253132s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654086841 (263h57m53.385354471s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654373080 (263h57m53.3856407s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.735474331 (263h57m53.466741961s) [upload=274521]: Next returned nil segment Aug 19 17:24:12.735486304 (263h57m53.466753934s) [upload=274521]: Waiting for error group managing segments... Aug 19 17:24:12.756883830 (263h57m53.48815145s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.766603935 (263h57m53.497871585s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.809581250 (263h57m53.540859941s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.908061427 (263h57m53.639329057s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.147463951 (263h57m53.878731601s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.364439955 (263h57m54.095707575s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:51.706547376 (263h58m32.437815006s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:52.370283683 (263h58m33.101551303s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.816229069 (263h58m36.547496699s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.944464809 (263h58m36.675732439s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.083334942 (263h58m48.814602572s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.405488480 (263h58m49.136756101s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.160489383 (263h58m52.891757013s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.595037822 (263h58m53.326305452s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:12tQRqowH4aDr9h3s1WLs1o7na1ueiafwgUeeHfAB6g7mcE6e1t, address:45.91.116.14:4213): protocol: rpc: dial tcp 45.91.116.14:4213: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595089570 (263h58m53.326357189s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1KvoMM3ptQXSxPhXFe3UoS145h8n3DMT9qmeaoR3bqNa8KLMm9, address:45.91.116.12:4250): protocol: write tcp 5.161.89.252:56044->45.91.116.12:4250: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595102303 (263h58m53.326369933s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1yLCMLdMxDT9HviM2QRVvP1hkektw7WmGxm8QVpp7h7ZVexkjy, address:45.91.116.4:4950): protocol: rpc: dial tcp 45.91.116.4:4950: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595112262 (263h58m53.326379892s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1pX3YNcCwaUeMstL4NNwvnKQmHbmdxLTUc8sDHwyvZGZtzmYgW, address:77.90.31.11:4574): protocol: write tcp 5.161.89.252:9456->77.90.31.11:4574: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595145234 (263h58m53.326412864s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1gnJixpV2uZqEMTxi22y1EAzKCHXaQeBWakhT6AcEkrENWT8CT, address:45.91.116.13:4182): protocol: write tcp 5.161.89.252:9502->45.91.116.13:4182: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595163289 (263h58m53.326430909s) [seg_pos={0 0}, upload=274521]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 19 17:25:12.636371896 (263h58m53.367639527s) [upload=274521]: Done uploading segments... Aug 19 17:25:12.636391424 (263h58m53.367659054s) [upload=274521]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 551.695114ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 554.83969ms FAIL (2025-08-19 17:32:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 554.859668ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X...X..........] duration: 356.489131ms FAIL (2025-08-19 17:32:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 356.504689ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X....................................X.X.X...........] duration: 501.472312ms FAIL (2025-08-19 17:32:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 19 17:24:12.613365846 (263h57m53.344633476s) [upload=274521]: Starting upload Aug 19 17:24:12.613406413 (263h57m53.344674043s) [upload=274521]: Done starting upload Aug 19 17:24:12.613493606 (263h57m53.344761226s) [upload=274521]: Uploading segments... Aug 19 17:24:12.613734899 (263h57m53.345002529s) [upload=274521]: Got next segment. Inline: false Aug 19 17:24:12.613764896 (263h57m53.345032526s) [upload=274521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:24:12.613769284 (263h57m53.345036914s) [upload=274521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:24:12.651951109 (263h57m53.383218739s) [seg_pos={0 0}, upload=274521]: Begin upload segment... Aug 19 17:24:12.651990573 (263h57m53.383258193s) [seg_pos={0 0}, upload=274521]: Done begin upload segment. Aug 19 17:24:12.652011803 (263h57m53.383279433s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652035357 (263h57m53.383302987s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652327476 (263h57m53.383595106s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652383030 (263h57m53.38365066s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652565463 (263h57m53.383833093s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652676883 (263h57m53.383944513s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652812508 (263h57m53.384080138s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652949715 (263h57m53.384217345s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653091241 (263h57m53.384358871s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653251271 (263h57m53.384518892s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653305264 (263h57m53.384572884s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653509577 (263h57m53.384777207s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653518354 (263h57m53.384785984s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653743006 (263h57m53.385010626s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653788721 (263h57m53.385056351s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653985512 (263h57m53.385253132s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654086841 (263h57m53.385354471s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654373080 (263h57m53.3856407s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.735474331 (263h57m53.466741961s) [upload=274521]: Next returned nil segment Aug 19 17:24:12.735486304 (263h57m53.466753934s) [upload=274521]: Waiting for error group managing segments... Aug 19 17:24:12.756883830 (263h57m53.48815145s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.766603935 (263h57m53.497871585s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.809581250 (263h57m53.540859941s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.908061427 (263h57m53.639329057s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.147463951 (263h57m53.878731601s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.364439955 (263h57m54.095707575s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:51.706547376 (263h58m32.437815006s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:52.370283683 (263h58m33.101551303s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.816229069 (263h58m36.547496699s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.944464809 (263h58m36.675732439s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.083334942 (263h58m48.814602572s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.405488480 (263h58m49.136756101s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.160489383 (263h58m52.891757013s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.595037822 (263h58m53.326305452s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:12tQRqowH4aDr9h3s1WLs1o7na1ueiafwgUeeHfAB6g7mcE6e1t, address:45.91.116.14:4213): protocol: rpc: dial tcp 45.91.116.14:4213: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595089570 (263h58m53.326357189s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1KvoMM3ptQXSxPhXFe3UoS145h8n3DMT9qmeaoR3bqNa8KLMm9, address:45.91.116.12:4250): protocol: write tcp 5.161.89.252:56044->45.91.116.12:4250: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595102303 (263h58m53.326369933s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1yLCMLdMxDT9HviM2QRVvP1hkektw7WmGxm8QVpp7h7ZVexkjy, address:45.91.116.4:4950): protocol: rpc: dial tcp 45.91.116.4:4950: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595112262 (263h58m53.326379892s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1pX3YNcCwaUeMstL4NNwvnKQmHbmdxLTUc8sDHwyvZGZtzmYgW, address:77.90.31.11:4574): protocol: write tcp 5.161.89.252:9456->77.90.31.11:4574: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595145234 (263h58m53.326412864s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1gnJixpV2uZqEMTxi22y1EAzKCHXaQeBWakhT6AcEkrENWT8CT, address:45.91.116.13:4182): protocol: write tcp 5.161.89.252:9502->45.91.116.13:4182: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595163289 (263h58m53.326430909s) [seg_pos={0 0}, upload=274521]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 19 17:25:12.636371896 (263h58m53.367639527s) [upload=274521]: Done uploading segments... Aug 19 17:25:12.636391424 (263h58m53.367659054s) [upload=274521]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 501.495445ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 1.699751598s FAIL (2025-08-19 17:32:00 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.699833613s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X...........X...................] duration: 135.048142ms FAIL (2025-08-19 17:31:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 135.072097ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X....X..........] duration: 782.197607ms FAIL (2025-08-19 17:31:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 782.222926ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 474.69944ms FAIL (2025-08-19 17:31:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 474.727143ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 484.312856ms FAIL (2025-08-19 17:31:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 484.342452ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 1.344792199s FAIL (2025-08-19 17:31:20 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.344812567s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)