Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 395.520007ms FAIL (2025-08-19 18:21:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 395.545034ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 421.067896ms FAIL (2025-08-19 18:20:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 421.092271ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 491.210716ms FAIL (2025-08-19 18:20:07 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 491.231375ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 398.693439ms FAIL (2025-08-19 18:19:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 398.714958ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 214.844858ms FAIL (2025-08-19 18:19:01 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 214.866389ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 409.298698ms FAIL (2025-08-19 18:18:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 409.317804ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 204.949684ms FAIL (2025-08-19 18:18:06 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 205.007743ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 438.29741ms FAIL (2025-08-19 18:17:45 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 438.323358ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 211.953885ms FAIL (2025-08-19 18:16:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 211.982449ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 402.265599ms FAIL (2025-08-19 18:16:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 402.295676ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 263.433938ms FAIL (2025-08-19 18:16:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 263.485876ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 474.026702ms FAIL (2025-08-19 18:15:52 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 474.042421ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 217.588327ms FAIL (2025-08-19 18:15:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 217.617131ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 393.567849ms FAIL (2025-08-19 18:15:13 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 393.602113ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 342.813581ms FAIL (2025-08-19 18:15:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 342.870569ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 435.68837ms FAIL (2025-08-19 18:14:29 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 435.712156ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 455.068069ms FAIL (2025-08-19 18:14:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 455.087285ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 502.003157ms FAIL (2025-08-19 18:14:13 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 502.034857ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 550.152911ms FAIL (2025-08-19 18:14:01 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 550.176474ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 309.490726ms FAIL (2025-08-19 18:13:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 309.517126ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 525.321297ms FAIL (2025-08-19 18:13:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 525.341415ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 187.363902ms FAIL (2025-08-19 18:13:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 187.388169ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 532.127519ms FAIL (2025-08-19 18:13:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 532.14961ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [X.............XX............................................] duration: 454.347453ms FAIL (2025-08-19 18:12:46 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 454.374214ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 371.282382ms FAIL (2025-08-19 18:12:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 371.298832ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 210.580064ms FAIL (2025-08-19 18:12:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 210.604208ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 362.291112ms FAIL (2025-08-19 18:12:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 362.308775ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X...X.......................................................] duration: 388.958456ms FAIL (2025-08-19 18:11:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 388.996226ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [.X.............XX...........................................] duration: 552.347696ms FAIL (2025-08-19 18:11:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 552.369508ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.X.............XX..........................................] duration: 358.231531ms FAIL (2025-08-19 18:11:04 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 358.247591ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 243.743737ms FAIL (2025-08-19 18:11:02 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 243.762152ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X...X......................................................] duration: 358.301394ms FAIL (2025-08-19 18:10:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 358.319557ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 159.289656ms FAIL (2025-08-19 18:10:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 159.306427ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.X.............XX.........................................] duration: 549.818963ms FAIL (2025-08-19 18:10:20 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 549.843119ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X...X.....................................................] duration: 378.717818ms FAIL (2025-08-19 18:10:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 378.733728ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.X.X.............XX........................................] duration: 447.901947ms FAIL (2025-08-19 18:09:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 447.928427ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 369.75982ms FAIL (2025-08-19 18:09:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 369.782563ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 166.572986ms FAIL (2025-08-19 18:09:29 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 166.598514ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X...X....................................................] duration: 459.314398ms FAIL (2025-08-19 18:09:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.334105ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 391.324825ms FAIL (2025-08-19 18:08:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 391.341577ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.X.X.............XX.......................................] duration: 775.947181ms FAIL (2025-08-19 18:08:25 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 775.971686ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 207.433096ms FAIL (2025-08-19 18:08:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 207.450389ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 350.548611ms FAIL (2025-08-19 18:08:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 350.564421ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X...X...................................................] duration: 392.681085ms FAIL (2025-08-19 18:07:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 392.706441ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 222.95934ms FAIL (2025-08-19 18:07:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 222.977825ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.X.X.............XX......................................] duration: 447.250864ms FAIL (2025-08-19 18:07:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:35:24.598550906 (264h9m5.329818526s) [upload=274709]: Starting upload Aug 19 17:35:24.598598336 (264h9m5.329865966s) [upload=274709]: Done starting upload Aug 19 17:35:24.598705527 (264h9m5.329973177s) [upload=274709]: Uploading segments... Aug 19 17:35:24.598834049 (264h9m5.330101679s) [upload=274709]: Got next segment. Inline: false Aug 19 17:35:24.598891597 (264h9m5.330159247s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:35:24.598922305 (264h9m5.330189965s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:35:24.622781310 (264h9m5.354048961s) [seg_pos={0 0}, upload=274709]: Begin upload segment... Aug 19 17:35:24.622850871 (264h9m5.35411852s) [seg_pos={0 0}, upload=274709]: Done begin upload segment. Aug 19 17:35:24.622881097 (264h9m5.354148758s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.622915473 (264h9m5.354183093s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623242778 (264h9m5.354510398s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623434598 (264h9m5.354702257s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623465406 (264h9m5.354733026s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623704975 (264h9m5.354972595s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.623972749 (264h9m5.355240369s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624046368 (264h9m5.355314018s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624428025 (264h9m5.355695684s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.624838907 (264h9m5.356106557s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625410892 (264h9m5.356678542s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.625821152 (264h9m5.357088803s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626247935 (264h9m5.357515585s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.626718509 (264h9m5.35798616s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627108321 (264h9m5.358375972s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627468548 (264h9m5.358736198s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.627964190 (264h9m5.35923184s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.628364072 (264h9m5.359631721s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Uploading piece... Aug 19 17:35:24.722088727 (264h9m5.453356348s) [upload=274709]: Next returned nil segment Aug 19 17:35:24.722132710 (264h9m5.45340033s) [upload=274709]: Waiting for error group managing segments... Aug 19 17:35:24.742673821 (264h9m5.473941441s) [noise=true , address=104.36.82.60:4349 , piece_id=HFWGB436WWH535HPNQ7SLRJRQM7RJDQW4CEERUULHQCQ2ROQGWHA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.743213144 (264h9m5.474480765s) [noise=true , address=104.36.82.23:4476 , piece_id=SRG6T76SWPBTJ6AEHCA2QE44DGAKKDZGQQTQBQJAAMH5FNBJOOPQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.744830845 (264h9m5.476098485s) [noise=true , address=104.36.82.11:4123 , piece_id=UXFAZM72ULZJDE4WRGWOFIES6OIFNME5KRIPHH3HZ62GJDGBAJGA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.747023905 (264h9m5.478291526s) [noise=true , address=77.90.31.11:4566 , piece_id=4GVA4CW4YJGMSIDBG3RUGHJISQMBQVJG26ELKJLOJHJ5QWH42PZA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.753657211 (264h9m5.484924862s) [noise=true , address=104.36.82.15:4270 , piece_id=CRUIMZHVLJ4TSPJDQBNBF2DNLTAAINMYKWN7ANGLCUKCPXHRW54Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.754442816 (264h9m5.485710437s) [noise=true , address=104.36.82.12:4156 , piece_id=OM75LKW4ZTU7BLIN4IGLWGZMBA4KFUZ275LEPM3DVAZHHE2UGUKA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.756549285 (264h9m5.487816925s) [noise=true , address=45.91.116.12:4246 , piece_id=GCV2OSOQ2WOTMRXDC47HEFBEGFYEWRA5TV6BRBNR2JK6N3N6R7AQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.757132692 (264h9m5.488400312s) [noise=true , address=104.36.82.35:4607 , piece_id=CWGDXRAAOPBVZK3HM4MFZTC3DT2KBS7GCGK7QCZOQL6PL2S3IHDA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.758813881 (264h9m5.490081521s) [noise=true , address=77.90.31.13:4655 , piece_id=24S543QUBH4DQF3N2QDHHUSBOODABXVFD6LNVQJJHFDOZSHQFESQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.760553070 (264h9m5.491820689s) [noise=true , address=104.36.82.39:4728 , piece_id=H2EHJSSJLZUFRCDZORJCVRU2RENBGNYS4VLLNFP3CVLFPHWQSL2Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.761195576 (264h9m5.492463197s) [noise=true , address=45.91.116.13:4171 , piece_id=ZE654ZPDWEIAU3GG4DIC7GWF6QOKBKSTYB7MYCVJGQZFGNIC7OPA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.762297076 (264h9m5.493564726s) [noise=true , address=45.91.116.11:4131 , piece_id=QNET7XJBYHVXRBBP4YPIDMPGH3KNS3UM27D7YCC5QEYAG6XETWYQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.778979887 (264h9m5.510247517s) [noise=true , address=77.90.31.17:4796 , piece_id=77XR47AJGR5GUGFNQ3F7LVARB36B3MLPPJEVXH5IHS7KBGKTIH7Q, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785621458 (264h9m5.516889107s) [noise=true , address=104.36.82.19:4427 , piece_id=R4SVH2DTKSTPPREK3ZKI2CCLP4JLJQZLIHTIA5MJ4Q6VSGN56APA, seg_pos={0 0}, upload=274709]: Done uploading piece. err: <nil> Aug 19 17:35:24.785659460 (264h9m5.5169271s) [seg_pos={0 0}, upload=274709]: Segment reached optimal threshold of 14 pieces. Aug 19 17:35:24.785819780 (264h9m5.51708741s) [noise=true , address=77.90.31.15:4725 , piece_id=PLEAIIR264NPCGAX3FGZICI4WG7HJQ5LBMHOHDJ4WIWDCNFOLZSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1nstPr2GMJo6pnBmBuDME5QYw3mXgnqLoaTKkcKUhjSbAY8mQk): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.785925820 (264h9m5.51719347s) [noise=true , address=45.91.116.6:4005 , piece_id=WSW5RKYHGG56AMLIGSFYDS364NLBJMSV4XKKMKOSV6IGLQQCJ4NQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5pHzd3BFgkMPrMhSwTQBSxbSp7khERtcaCbZn1crJobN5zWn): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786017702 (264h9m5.517285332s) [noise=true , address=77.90.31.18:4808 , piece_id=FGDO53D4FLPXTKC2VAZWK5QRM4L53CQWTNCFXTXO4MFSNAZ5NOSQ, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1bAoTamwp534btqQDaG1TdCMJghkj8ZqTQtqJub1X3kp3VoJn9): protocol: expected piece hash; context canceled; EOF Aug 19 17:35:24.786091801 (264h9m5.517359452s) [noise=true , address=66.23.193.175:4559 , piece_id=C7LOCAYKD74IZCF7ZXPPHPQTLKO4J5BJQY7S3ICYFJRM6GLKJO2A, seg_pos={0 0}, upload=274709]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ZedZCGumnM48qSS4jd68SuW45Q5SMqxY4UZvLrBkoWX4i6syp): protocol: read tcp 5.161.89.252:57194->66.23.193.175:4559: use of closed network connection; context canceled Aug 19 17:35:24.786125735 (264h9m5.517393385s) [seg_pos={0 0}, upload=274709]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 17:35:24.786208471 (264h9m5.517476122s) [upload=274709]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 17:35:24.786213561 (264h9m5.517481211s) [upload=274709]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:35:24.829790861 (264h9m5.56105848s) [upload=274709]: Done uploading segments... Aug 19 17:35:24.829804556 (264h9m5.561072186s) [upload=274709]: Upload finished. err: <nil> Duration: 447.276042ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 520.47713ms FAIL (2025-08-19 18:06:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 520.501596ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X...X..................................................] duration: 365.755245ms FAIL (2025-08-19 18:06:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 365.773309ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 435.68688ms FAIL (2025-08-19 18:06:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 435.710945ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 646.93208ms FAIL (2025-08-19 18:06:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== Aug 19 17:34:41.705413405 (264h8m22.436681035s) [upload=274699]: Starting upload Aug 19 17:34:41.705456125 (264h8m22.436723755s) [upload=274699]: Done starting upload Aug 19 17:34:41.705499847 (264h8m22.436767467s) [upload=274699]: Uploading segments... Aug 19 17:34:41.705540113 (264h8m22.436807743s) [upload=274699]: Got next segment. Inline: true Aug 19 17:34:41.705546515 (264h8m22.436814165s) [upload=274699]: Waiting for error group managing segments... Aug 19 17:34:41.705593683 (264h8m22.436861313s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:34:41.705597821 (264h8m22.436865451s) [upload=274699]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:34:41.705600336 (264h8m22.436867955s) [upload=274699]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:34:41.781184293 (264h8m22.512451923s) [upload=274699]: Done uploading segments... Aug 19 17:34:41.781200663 (264h8m22.512468293s) [upload=274699]: Upload finished. err: <nil> Duration:* 646.95293ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)