Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 3.749672529s FAIL (2025-08-19 17:25:44 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 3.749687457s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 639.511103ms FAIL (2025-08-19 17:25:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 639.526212ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X....................................X.X.X.] duration: 1.27990815s FAIL (2025-08-19 17:25:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 19 17:24:12.613365846 (263h57m53.344633476s) [upload=274521]: Starting upload Aug 19 17:24:12.613406413 (263h57m53.344674043s) [upload=274521]: Done starting upload Aug 19 17:24:12.613493606 (263h57m53.344761226s) [upload=274521]: Uploading segments... Aug 19 17:24:12.613734899 (263h57m53.345002529s) [upload=274521]: Got next segment. Inline: false Aug 19 17:24:12.613764896 (263h57m53.345032526s) [upload=274521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:24:12.613769284 (263h57m53.345036914s) [upload=274521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:24:12.651951109 (263h57m53.383218739s) [seg_pos={0 0}, upload=274521]: Begin upload segment... Aug 19 17:24:12.651990573 (263h57m53.383258193s) [seg_pos={0 0}, upload=274521]: Done begin upload segment. Aug 19 17:24:12.652011803 (263h57m53.383279433s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652035357 (263h57m53.383302987s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652327476 (263h57m53.383595106s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652383030 (263h57m53.38365066s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652565463 (263h57m53.383833093s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652676883 (263h57m53.383944513s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652812508 (263h57m53.384080138s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652949715 (263h57m53.384217345s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653091241 (263h57m53.384358871s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653251271 (263h57m53.384518892s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653305264 (263h57m53.384572884s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653509577 (263h57m53.384777207s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653518354 (263h57m53.384785984s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653743006 (263h57m53.385010626s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653788721 (263h57m53.385056351s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653985512 (263h57m53.385253132s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654086841 (263h57m53.385354471s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654373080 (263h57m53.3856407s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.735474331 (263h57m53.466741961s) [upload=274521]: Next returned nil segment Aug 19 17:24:12.735486304 (263h57m53.466753934s) [upload=274521]: Waiting for error group managing segments... Aug 19 17:24:12.756883830 (263h57m53.48815145s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.766603935 (263h57m53.497871585s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.809581250 (263h57m53.540859941s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.908061427 (263h57m53.639329057s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.147463951 (263h57m53.878731601s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.364439955 (263h57m54.095707575s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:51.706547376 (263h58m32.437815006s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:52.370283683 (263h58m33.101551303s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.816229069 (263h58m36.547496699s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.944464809 (263h58m36.675732439s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.083334942 (263h58m48.814602572s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.405488480 (263h58m49.136756101s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.160489383 (263h58m52.891757013s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.595037822 (263h58m53.326305452s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:12tQRqowH4aDr9h3s1WLs1o7na1ueiafwgUeeHfAB6g7mcE6e1t, address:45.91.116.14:4213): protocol: rpc: dial tcp 45.91.116.14:4213: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595089570 (263h58m53.326357189s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1KvoMM3ptQXSxPhXFe3UoS145h8n3DMT9qmeaoR3bqNa8KLMm9, address:45.91.116.12:4250): protocol: write tcp 5.161.89.252:56044->45.91.116.12:4250: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595102303 (263h58m53.326369933s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1yLCMLdMxDT9HviM2QRVvP1hkektw7WmGxm8QVpp7h7ZVexkjy, address:45.91.116.4:4950): protocol: rpc: dial tcp 45.91.116.4:4950: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595112262 (263h58m53.326379892s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1pX3YNcCwaUeMstL4NNwvnKQmHbmdxLTUc8sDHwyvZGZtzmYgW, address:77.90.31.11:4574): protocol: write tcp 5.161.89.252:9456->77.90.31.11:4574: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595145234 (263h58m53.326412864s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1gnJixpV2uZqEMTxi22y1EAzKCHXaQeBWakhT6AcEkrENWT8CT, address:45.91.116.13:4182): protocol: write tcp 5.161.89.252:9502->45.91.116.13:4182: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595163289 (263h58m53.326430909s) [seg_pos={0 0}, upload=274521]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 19 17:25:12.636371896 (263h58m53.367639527s) [upload=274521]: Done uploading segments... Aug 19 17:25:12.636391424 (263h58m53.367659054s) [upload=274521]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1.279923059s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................X....................................X.X.X] duration: 1m0.042668787s FAIL (2025-08-19 17:25:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== Aug 19 17:24:12.613365846 (263h57m53.344633476s) [upload=274521]: Starting upload Aug 19 17:24:12.613406413 (263h57m53.344674043s) [upload=274521]: Done starting upload Aug 19 17:24:12.613493606 (263h57m53.344761226s) [upload=274521]: Uploading segments... Aug 19 17:24:12.613734899 (263h57m53.345002529s) [upload=274521]: Got next segment. Inline: false Aug 19 17:24:12.613764896 (263h57m53.345032526s) [upload=274521]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:24:12.613769284 (263h57m53.345036914s) [upload=274521]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:24:12.651951109 (263h57m53.383218739s) [seg_pos={0 0}, upload=274521]: Begin upload segment... Aug 19 17:24:12.651990573 (263h57m53.383258193s) [seg_pos={0 0}, upload=274521]: Done begin upload segment. Aug 19 17:24:12.652011803 (263h57m53.383279433s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652035357 (263h57m53.383302987s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652327476 (263h57m53.383595106s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652383030 (263h57m53.38365066s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652565463 (263h57m53.383833093s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652676883 (263h57m53.383944513s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652812508 (263h57m53.384080138s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.652949715 (263h57m53.384217345s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653091241 (263h57m53.384358871s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653251271 (263h57m53.384518892s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653305264 (263h57m53.384572884s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653509577 (263h57m53.384777207s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653518354 (263h57m53.384785984s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653743006 (263h57m53.385010626s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653788721 (263h57m53.385056351s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.653985512 (263h57m53.385253132s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654086841 (263h57m53.385354471s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.654373080 (263h57m53.3856407s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Uploading piece... Aug 19 17:24:12.735474331 (263h57m53.466741961s) [upload=274521]: Next returned nil segment Aug 19 17:24:12.735486304 (263h57m53.466753934s) [upload=274521]: Waiting for error group managing segments... Aug 19 17:24:12.756883830 (263h57m53.48815145s) [noise=true , address=104.36.82.18:4381 , piece_id=RKK7ZUWNENGDCVUT5WG7BDDWV27PI3HXXKLAIFBTBCURFEMAYWPA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.766603935 (263h57m53.497871585s) [noise=true , address=104.36.82.45:4911 , piece_id=ZXUDCVWZU52QB5GD5CRK3KECCMLBNDSJSAE3ODVHR7CYRDI4TDGA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.809581250 (263h57m53.540859941s) [noise=true , address=104.36.82.37:4654 , piece_id=YHZ2ZZLFZIL6WU6SQFABTJYWLO7Q3IKQ6YRHV5O66T2WDULFX4LQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:12.908061427 (263h57m53.639329057s) [noise=true , address=79.127.201.215:4354 , piece_id=ILHYGB4D4OC5ZLMIQMOAP2VBUTPPZP73GSAWTWBVQCYD7UFCJ5ZA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.147463951 (263h57m53.878731601s) [noise=true , address=79.127.205.246:4258 , piece_id=62LLK7AA6RSVT6DGM6OBIMQVSTSVL7CKVOLSA4SA4UXBHRWINPOQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:13.364439955 (263h57m54.095707575s) [noise=true , address=64.49.13.26:4950 , piece_id=DF2W5SXMRPYVOO3VYRBOUICKMLALFEUO44RIRXHPXBINUUQ2KOHQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:51.706547376 (263h58m32.437815006s) [noise=true , address=77.90.31.12:4610 , piece_id=KGPE5FMXNSZBCXFT2RICGZBMN6NBHCDHNL5DAMJ4ERCKJTNHJYRA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:52.370283683 (263h58m33.101551303s) [noise=true , address=77.90.31.14:4672 , piece_id=S7LZJKYJYODOSPDQTY3IGUZDAUEV3FP2NOJ4YDJYPLP5OLGXIYXQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.816229069 (263h58m36.547496699s) [noise=true , address=77.90.31.18:4837 , piece_id=N7L4YWBPVUTRQGYK7DP6YRFXUTV4PPCK6ZP3PU3WTVYJOVKEEFQQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:24:55.944464809 (263h58m36.675732439s) [noise=true , address=77.90.31.16:4747 , piece_id=O7DENBKJSUM475ACFXOZ76ZNHPTKPKNVWRI5EODS4NSV7QFK5JCA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.083334942 (263h58m48.814602572s) [noise=true , address=45.91.116.10:4089 , piece_id=3COZWVBKUODK4BY2ZRZHWDF5Z2LQRJMT7WWDEEK5U24R7GHMEMFA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:08.405488480 (263h58m49.136756101s) [noise=true , address=45.91.116.11:4130 , piece_id=I3YU33P2NBUHDO25RRJAZA7Q4PBEKHGS6L2LS6QU4PZ54N5FYRWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.160489383 (263h58m52.891757013s) [noise=true , address=45.91.116.8:4046 , piece_id=H5JXXHFSR3A6FCZHHGZMQNYNSHME32X2SJ4B2CFRPCJFFYH4XVGQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: <nil> Aug 19 17:25:12.595037822 (263h58m53.326305452s) [noise=true , address=45.91.116.14:4213 , piece_id=M44PJHBENOPUNW7Y36ALXKWWSYSNKIVCWMMRPUSTQARFIUBHSSFQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:12tQRqowH4aDr9h3s1WLs1o7na1ueiafwgUeeHfAB6g7mcE6e1t, address:45.91.116.14:4213): protocol: rpc: dial tcp 45.91.116.14:4213: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595089570 (263h58m53.326357189s) [noise=true , address=45.91.116.12:4250 , piece_id=T3LKOF56LCVHNCFEFEPJWOW5SV6CYPASBLIMOEEMG7SCDITQ6DLQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1KvoMM3ptQXSxPhXFe3UoS145h8n3DMT9qmeaoR3bqNa8KLMm9, address:45.91.116.12:4250): protocol: write tcp 5.161.89.252:56044->45.91.116.12:4250: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595102303 (263h58m53.326369933s) [noise=true , address=45.91.116.4:4950 , piece_id=AHBPJ5MQGTIZATXP6WMQK7BB3FDWKDWNONTBVICHCBJLIK6VIDTA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1yLCMLdMxDT9HviM2QRVvP1hkektw7WmGxm8QVpp7h7ZVexkjy, address:45.91.116.4:4950): protocol: rpc: dial tcp 45.91.116.4:4950: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595112262 (263h58m53.326379892s) [noise=true , address=77.90.31.11:4574 , piece_id=TXNGYH7XVLWTNFITEVSI4LY23T6WBQT4OLFGGXWPLPY57KW4KZWA, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1pX3YNcCwaUeMstL4NNwvnKQmHbmdxLTUc8sDHwyvZGZtzmYgW, address:77.90.31.11:4574): protocol: write tcp 5.161.89.252:9456->77.90.31.11:4574: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595145234 (263h58m53.326412864s) [noise=true , address=45.91.116.13:4182 , piece_id=GHBD43AKTIKHGLNGIIIBYDIT3RLEWI226TWIWGH5TVSOQHLQ4QIQ, seg_pos={0 0}, upload=274521]: Done uploading piece. err: ecclient: upload failed (node:1gnJixpV2uZqEMTxi22y1EAzKCHXaQeBWakhT6AcEkrENWT8CT, address:45.91.116.13:4182): protocol: write tcp 5.161.89.252:9502->45.91.116.13:4182: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:25:12.595163289 (263h58m53.326430909s) [seg_pos={0 0}, upload=274521]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Aug 19 17:25:12.636371896 (263h58m53.367639527s) [upload=274521]: Done uploading segments... Aug 19 17:25:12.636391424 (263h58m53.367659054s) [upload=274521]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.042684898s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X...........X........] duration: 277.328553ms FAIL (2025-08-19 17:25:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 277.345956ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......XX..............................................X...X.] duration: 439.139505ms FAIL (2025-08-19 17:24:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 439.15819ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 10.588379058s FAIL (2025-08-19 17:24:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 10.588400469s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X....X..............................................X....X.] duration: 7.817404939s FAIL (2025-08-19 17:24:49 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 7.817435356s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X....X..............................................X....X] duration: 30.019916136s FAIL (2025-08-19 17:24:41 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=274520]: Done begin upload segment. Aug 19 17:24:12.024531859 (263h57m52.755799509s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024547178 (263h57m52.755814828s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024953521 (263h57m52.756221171s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.024997523 (263h57m52.756265174s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025367439 (263h57m52.756635089s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025414347 (263h57m52.756681997s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025761920 (263h57m52.75702958s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.025888298 (263h57m52.757155948s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026138559 (263h57m52.757406208s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026283681 (263h57m52.75755133s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026491962 (263h57m52.757759613s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026640120 (263h57m52.757907771s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026842701 (263h57m52.758110352s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.026995178 (263h57m52.758262828s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027210582 (263h57m52.758478232s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.027858310 (263h57m52.75912596s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Uploading piece... Aug 19 17:24:12.143746757 (263h57m52.875014407s) [upload=274520]: Next returned nil segment Aug 19 17:24:12.143775502 (263h57m52.875043152s) [upload=274520]: Waiting for error group managing segments... Aug 19 17:24:12.157847553 (263h57m52.889115213s) [noise=true , address=104.36.82.13:4185 , piece_id=G6V7LQWJ3UC4PMMPBZWE67KWQXQENEEO4O4763VEGVGEVIDJ5FEA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158547338 (263h57m52.889814989s) [noise=true , address=104.36.82.5:4837 , piece_id=AM2AXREOT4ZII77PAWUE76BM6AR4YGI3XF5CGUWRKEE5R5FTDIFA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.158808157 (263h57m52.890075818s) [noise=true , address=104.36.82.2:4729 , piece_id=LY2SWRDWXQHES5FTAF5CP6KCNVCY6FHIBKJNLOCEFYSBNBPIPNIA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.159159038 (263h57m52.890426688s) [noise=true , address=104.36.82.7:4912 , piece_id=ADZS3KGOOF4HPEYRXYTTS7WHDRKRBGRIKUKCORRSAENYPHXMEGHA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.160381044 (263h57m52.891648694s) [noise=true , address=104.36.82.7:4909 , piece_id=QB6Z7VE6VFC5L3BNINLJTO53YACDUGNBL6U7WUHKSG5HAFVSXAJQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:12.163470309 (263h57m52.894737959s) [noise=true , address=104.36.82.9:4007 , piece_id=YEAQAGKXBODCVNIAG2S7TT5MP3TKTC5PAWGA7CIIBEW5DYS7UQCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: <nil> Aug 19 17:24:41.979520414 (263h58m22.710788064s) [noise=true , address=77.90.31.13:4628 , piece_id=B2BESUZXB37NOFSOEUPCUXX3AMEZ44S62HIKN7K4P2FD65CPEDPQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:36766->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980102297 (263h58m22.711369958s) [noise=true , address=45.91.116.14:4190 , piece_id=3AZCPPFTZFZ7ANINXPAIR433CKJUXEVDB7FYJIQDLHMYIMRFTGKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:21646->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980158464 (263h58m22.711426114s) [noise=true , address=45.91.116.4:4939 , piece_id=BTZOJWID4G2ADAZKPQ6PW7SMV42R2FM756KFMCBSBAMFMJDFULWQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:51578->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980204240 (263h58m22.71147189s) [noise=true , address=77.90.31.11:4554 , piece_id=OQYJJPQRLA64VFXA5VBWDKJ73Q2YMLTRW3UXJVNBQ3BLCZ55TZ5Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek, address:77.90.31.11:4554): protocol: read tcp 5.161.89.252:9778->77.90.31.11:4554: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980249345 (263h58m22.711517004s) [noise=true , address=77.90.31.14:4661 , piece_id=WDR237S6GBHWJGAO2L7M5YASTXX6QOLMIMDZM2D57PQHZA7BYB2Q, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:22472->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980292254 (263h58m22.711559905s) [noise=true , address=45.91.116.14:4189 , piece_id=QVER7IKF6T66GHOKDYVUVAYBF4OWORQV3IPKXKTREBYXOLKUI6HA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:34124->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980365091 (263h58m22.711632752s) [noise=true , address=45.91.116.8:4045 , piece_id=X2HZK7QLI7RRZMF6WYSHK4TYUCPWMF7XRY6KVWKG7SOMGZK4HZXA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:8238->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980403213 (263h58m22.711670863s) [noise=true , address=77.90.31.19:4841 , piece_id=MY2CTXR6M4Z7JDC5BE3HENPMUOXXSVX22PNG236QOH5F3552PFCQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: rpc: dial tcp 77.90.31.19:4841: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980547475 (263h58m22.711815124s) [noise=true , address=45.91.116.6:4973 , piece_id=MP533ZUEZDRXYTQPWKM5URFX6BB3WPGAPFHFMUJQVNBZFPTJHICA, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:29404->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980632083 (263h58m22.711899743s) [noise=true , address=77.90.31.12:4592 , piece_id=LYEZMNLTZVT4UQHZL53CGXZXPHLPJPO34EMLPPRARNVULCOQPXKQ, seg_pos={0 0}, upload=274520]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:24:41.980736129 (263h58m22.712003779s) [seg_pos={0 0}, upload=274520]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Aug 19 17:24:41.997898729 (263h58m22.729166379s) [upload=274520]: Done uploading segments... Aug 19 17:24:41.997919457 (263h58m22.729187108s) [upload=274520]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.019938057s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......XX..............................................X...X] duration: 30.002310995s FAIL (2025-08-19 17:24:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002329259s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 1.719062595s FAIL (2025-08-19 17:24:33 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.719083545s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 1.325065535s FAIL (2025-08-19 17:23:59 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.325082046s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [........................................X...........X.......] duration: 125.560001ms FAIL (2025-08-19 17:23:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 125.582964ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 449.363765ms FAIL (2025-08-19 17:23:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 449.387509ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................X....................................X.X.] duration: 656.957268ms FAIL (2025-08-19 17:23:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:22:30.199126700 (263h56m10.930394361s) [upload=274493]: Starting upload Aug 19 17:22:30.199223742 (263h56m10.930491392s) [upload=274493]: Done starting upload Aug 19 17:22:30.199282323 (263h56m10.930549974s) [upload=274493]: Uploading segments... Aug 19 17:22:30.199604057 (263h56m10.930871718s) [upload=274493]: Got next segment. Inline: false Aug 19 17:22:30.199649072 (263h56m10.930916732s) [upload=274493]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:22:30.199657277 (263h56m10.930924937s) [upload=274493]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:22:30.221225153 (263h56m10.952492803s) [upload=274493]: Next segment err: metaclient: Too Many Requests Aug 19 17:22:30.221240732 (263h56m10.952508382s) [upload=274493]: Done uploading segments... Aug 19 17:22:30.221255199 (263h56m10.952522849s) [upload=274493]: Upload finished. err: metaclient: Too Many Requests Duration: 656.978027ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X....X..............................................X....] duration: 783.264235ms FAIL (2025-08-19 17:23:27 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== Aug 19 17:20:25.233844508 (263h54m5.965112158s) [upload=274454]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:20:25.233849798 (263h54m5.965117448s) [upload=274454]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:25.340764042 (263h54m6.072031652s) [seg_pos={0 0}, upload=274454]: Begin upload segment... Aug 19 17:20:25.340793026 (263h54m6.072060646s) [seg_pos={0 0}, upload=274454]: Done begin upload segment. Aug 19 17:20:25.340821350 (263h54m6.072089041s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.340914916 (263h54m6.072182546s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341136623 (263h54m6.072404243s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341247761 (263h54m6.072515381s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341347639 (263h54m6.072615258s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341528018 (263h54m6.072795638s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341535622 (263h54m6.072803242s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341763621 (263h54m6.073031241s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341869960 (263h54m6.07313759s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342030262 (263h54m6.073297892s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342163261 (263h54m6.073430892s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342283257 (263h54m6.073550887s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342440372 (263h54m6.073708002s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342509302 (263h54m6.073776921s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342679041 (263h54m6.073946671s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.343286362 (263h54m6.074553981s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.429300735 (263h54m6.160568365s) [upload=274454]: Next returned nil segment Aug 19 17:20:25.429315603 (263h54m6.160583233s) [upload=274454]: Waiting for error group managing segments... Aug 19 17:20:25.442554419 (263h54m6.173822049s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.442663975 (263h54m6.173931605s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.443726752 (263h54m6.174994382s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.445744032 (263h54m6.177011652s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.459719312 (263h54m6.190986932s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.461728297 (263h54m6.192995927s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.467426444 (263h54m6.198694074s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:55.218201378 (263h54m35.949469008s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218314731 (263h54m35.949582361s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 5.161.89.252:23356->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218363783 (263h54m35.949631413s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: rpc: dial tcp 45.91.116.12:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218371297 (263h54m35.949638927s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218492334 (263h54m35.949759954s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218527600 (263h54m35.949795221s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218553008 (263h54m35.949820628s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 5.161.89.252:58664->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218563387 (263h54m35.949831008s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 5.161.89.252:21248->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218578345 (263h54m35.949845966s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 5.161.89.252:57908->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218608202 (263h54m35.949875823s) [seg_pos={0 0}, upload=274454]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 19 17:20:55.232496830 (263h54m35.96376445s) [upload=274454]: Done uploading segments... Aug 19 17:20:55.232513381 (263h54m35.963781001s) [upload=274454]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 783.289742ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........XX..............................................X...] duration: 790.912536ms FAIL (2025-08-19 17:23:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 790.927264ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 1.464876532s FAIL (2025-08-19 17:23:13 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.464890978s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 433.507632ms FAIL (2025-08-19 17:22:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 433.530114ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X...........X......] duration: 145.920118ms FAIL (2025-08-19 17:22:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 145.940035ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 1.663410996s FAIL (2025-08-19 17:22:50 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.663430442s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....X....X..............................................X...] duration: 390.428795ms FAIL (2025-08-19 17:22:34 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== Aug 19 17:20:25.233844508 (263h54m5.965112158s) [upload=274454]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:20:25.233849798 (263h54m5.965117448s) [upload=274454]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:25.340764042 (263h54m6.072031652s) [seg_pos={0 0}, upload=274454]: Begin upload segment... Aug 19 17:20:25.340793026 (263h54m6.072060646s) [seg_pos={0 0}, upload=274454]: Done begin upload segment. Aug 19 17:20:25.340821350 (263h54m6.072089041s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.340914916 (263h54m6.072182546s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341136623 (263h54m6.072404243s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341247761 (263h54m6.072515381s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341347639 (263h54m6.072615258s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341528018 (263h54m6.072795638s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341535622 (263h54m6.072803242s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341763621 (263h54m6.073031241s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341869960 (263h54m6.07313759s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342030262 (263h54m6.073297892s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342163261 (263h54m6.073430892s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342283257 (263h54m6.073550887s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342440372 (263h54m6.073708002s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342509302 (263h54m6.073776921s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342679041 (263h54m6.073946671s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.343286362 (263h54m6.074553981s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.429300735 (263h54m6.160568365s) [upload=274454]: Next returned nil segment Aug 19 17:20:25.429315603 (263h54m6.160583233s) [upload=274454]: Waiting for error group managing segments... Aug 19 17:20:25.442554419 (263h54m6.173822049s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.442663975 (263h54m6.173931605s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.443726752 (263h54m6.174994382s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.445744032 (263h54m6.177011652s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.459719312 (263h54m6.190986932s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.461728297 (263h54m6.192995927s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.467426444 (263h54m6.198694074s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:55.218201378 (263h54m35.949469008s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218314731 (263h54m35.949582361s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 5.161.89.252:23356->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218363783 (263h54m35.949631413s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: rpc: dial tcp 45.91.116.12:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218371297 (263h54m35.949638927s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218492334 (263h54m35.949759954s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218527600 (263h54m35.949795221s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218553008 (263h54m35.949820628s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 5.161.89.252:58664->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218563387 (263h54m35.949831008s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 5.161.89.252:21248->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218578345 (263h54m35.949845966s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 5.161.89.252:57908->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218608202 (263h54m35.949875823s) [seg_pos={0 0}, upload=274454]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 19 17:20:55.232496830 (263h54m35.96376445s) [upload=274454]: Done uploading segments... Aug 19 17:20:55.232513381 (263h54m35.963781001s) [upload=274454]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 390.450747ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................X....................................X.X] duration: 38.800516ms FAIL (2025-08-19 17:22:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:22:30.199126700 (263h56m10.930394361s) [upload=274493]: Starting upload Aug 19 17:22:30.199223742 (263h56m10.930491392s) [upload=274493]: Done starting upload Aug 19 17:22:30.199282323 (263h56m10.930549974s) [upload=274493]: Uploading segments... Aug 19 17:22:30.199604057 (263h56m10.930871718s) [upload=274493]: Got next segment. Inline: false Aug 19 17:22:30.199649072 (263h56m10.930916732s) [upload=274493]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:22:30.199657277 (263h56m10.930924937s) [upload=274493]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:22:30.221225153 (263h56m10.952492803s) [upload=274493]: Next segment err: metaclient: Too Many Requests Aug 19 17:22:30.221240732 (263h56m10.952508382s) [upload=274493]: Done uploading segments... Aug 19 17:22:30.221255199 (263h56m10.952522849s) [upload=274493]: Upload finished. err: metaclient: Too Many Requests Duration: 38.822937ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....X....X..............................................X..] duration: 450.803963ms FAIL (2025-08-19 17:22:17 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== Aug 19 17:20:25.233844508 (263h54m5.965112158s) [upload=274454]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:20:25.233849798 (263h54m5.965117448s) [upload=274454]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:25.340764042 (263h54m6.072031652s) [seg_pos={0 0}, upload=274454]: Begin upload segment... Aug 19 17:20:25.340793026 (263h54m6.072060646s) [seg_pos={0 0}, upload=274454]: Done begin upload segment. Aug 19 17:20:25.340821350 (263h54m6.072089041s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.340914916 (263h54m6.072182546s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341136623 (263h54m6.072404243s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341247761 (263h54m6.072515381s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341347639 (263h54m6.072615258s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341528018 (263h54m6.072795638s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341535622 (263h54m6.072803242s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341763621 (263h54m6.073031241s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341869960 (263h54m6.07313759s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342030262 (263h54m6.073297892s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342163261 (263h54m6.073430892s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342283257 (263h54m6.073550887s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342440372 (263h54m6.073708002s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342509302 (263h54m6.073776921s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342679041 (263h54m6.073946671s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.343286362 (263h54m6.074553981s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.429300735 (263h54m6.160568365s) [upload=274454]: Next returned nil segment Aug 19 17:20:25.429315603 (263h54m6.160583233s) [upload=274454]: Waiting for error group managing segments... Aug 19 17:20:25.442554419 (263h54m6.173822049s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.442663975 (263h54m6.173931605s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.443726752 (263h54m6.174994382s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.445744032 (263h54m6.177011652s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.459719312 (263h54m6.190986932s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.461728297 (263h54m6.192995927s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.467426444 (263h54m6.198694074s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:55.218201378 (263h54m35.949469008s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218314731 (263h54m35.949582361s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 5.161.89.252:23356->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218363783 (263h54m35.949631413s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: rpc: dial tcp 45.91.116.12:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218371297 (263h54m35.949638927s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218492334 (263h54m35.949759954s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218527600 (263h54m35.949795221s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218553008 (263h54m35.949820628s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 5.161.89.252:58664->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218563387 (263h54m35.949831008s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 5.161.89.252:21248->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218578345 (263h54m35.949845966s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 5.161.89.252:57908->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218608202 (263h54m35.949875823s) [seg_pos={0 0}, upload=274454]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 19 17:20:55.232496830 (263h54m35.96376445s) [upload=274454]: Done uploading segments... Aug 19 17:20:55.232513381 (263h54m35.963781001s) [upload=274454]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 450.828419ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........XX..............................................X..] duration: 402.139581ms FAIL (2025-08-19 17:22:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 402.204232ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 1.505846999s FAIL (2025-08-19 17:22:07 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.505867146s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 728.729052ms FAIL (2025-08-19 17:21:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 728.784978ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.803189499s FAIL (2025-08-19 17:21:42 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.803206311s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................X....................................X.] duration: 886.261499ms FAIL (2025-08-19 17:21:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:20:47.692563605 (263h54m28.423831256s) [upload=274464]: Starting upload Aug 19 17:20:47.692640950 (263h54m28.42390861s) [upload=274464]: Done starting upload Aug 19 17:20:47.692773529 (263h54m28.424041199s) [upload=274464]: Uploading segments... Aug 19 17:20:47.693054106 (263h54m28.424321756s) [upload=274464]: Got next segment. Inline: false Aug 19 17:20:47.693104080 (263h54m28.42437173s) [upload=274464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:20:47.693110282 (263h54m28.424377932s) [upload=274464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:47.696037212 (263h54m28.427304862s) [upload=274464]: Next segment err: metaclient: Too Many Requests Aug 19 17:20:47.696062039 (263h54m28.427329688s) [upload=274464]: Done uploading segments... Aug 19 17:20:47.696076836 (263h54m28.427344487s) [upload=274464]: Upload finished. err: metaclient: Too Many Requests Duration: 886.286035ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X...........X.....] duration: 1.125101357s FAIL (2025-08-19 17:21:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 1.125119069s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 1.610895083s FAIL (2025-08-19 17:21:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 1.610922886s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.017324033s FAIL (2025-08-19 17:21:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== ]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:30.724709450 (263h54m11.4559771s) [seg_pos={0 0}, upload=274457]: Begin upload segment... Aug 19 17:20:30.724753462 (263h54m11.456021113s) [seg_pos={0 0}, upload=274457]: Done begin upload segment. Aug 19 17:20:30.724781936 (263h54m11.456049586s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.724797054 (263h54m11.456064674s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725166569 (263h54m11.456434189s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725324816 (263h54m11.456592476s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725415496 (263h54m11.456683126s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725643935 (263h54m11.456911565s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725817342 (263h54m11.457084992s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725853730 (263h54m11.45712136s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.725891440 (263h54m11.457159071s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726110873 (263h54m11.457378504s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726322932 (263h54m11.457590582s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726338811 (263h54m11.457606432s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.726711522 (263h54m11.457979172s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727114529 (263h54m11.458382178s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.727561679 (263h54m11.458829329s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728022655 (263h54m11.459290316s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728544566 (263h54m11.459812225s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.728932415 (263h54m11.460200435s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Uploading piece... Aug 19 17:20:30.813701018 (263h54m11.544968628s) [upload=274457]: Next returned nil segment Aug 19 17:20:30.813711287 (263h54m11.544978897s) [upload=274457]: Waiting for error group managing segments... Aug 19 17:20:30.826872026 (263h54m11.558139646s) [noise=true , address=104.36.82.23:4458 , piece_id=B6SB2PNKJIT5GLPS7Z2ZFPT7A2PXDFHHC3QHCH2MQ3I4WK7MPRSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.832633702 (263h54m11.563901332s) [noise=true , address=104.36.82.44:4882 , piece_id=FADDUTQAO3INVE2JAWVR4SQISS7DZPV7ROI2FHIDZED6VQCLPKKA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.834038051 (263h54m11.565305681s) [noise=true , address=79.127.219.47:4062 , piece_id=A6756Y5PYSEIPMBUN6SAQ7DXQNADFJS6ECMYKBKSH2MPK6MYUT6A, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.844605367 (263h54m11.575872987s) [noise=true , address=104.36.82.2:4764 , piece_id=CBWOVFNOYNAY5F462E5J4KNX5LLYKZQEEJUOLKXLPDCWWHOMWW2Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.864222729 (263h54m11.59549035s) [noise=true , address=104.36.82.17:4353 , piece_id=X4HQY47TKKQ72QPDBMYFK3JHFKPQVTF5BFSXWYNYGAELKBV75KOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.866711214 (263h54m11.597978835s) [noise=true , address=104.36.82.34:4559 , piece_id=J5RYVJTRFQG6REZAZJW7URTZQHMQON2SUP5QHDIK53VHV3KJCLSA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:30.882824352 (263h54m11.614091972s) [noise=true , address=104.36.82.43:4839 , piece_id=SNPCZ3JHY3V2TNJI2INTRZSBDVXVTFQHKOITBEVAAZ5PFOHPSZSQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:20:31.250301081 (263h54m11.981568741s) [noise=true , address=79.127.205.235:4692 , piece_id=OJJ2PG7H3IBPMW5PFWO4YLQ3KT2D5YFH2PSWCOX3CGUZRFFZWB4Q, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:11.439696042 (263h54m52.170963692s) [noise=true , address=77.90.31.17:4797 , piece_id=AVUG5MVC6F7JVM2AWXCWFG5WZ7XGYT24H5QY36W7DCQQAMSHUTLA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:13.585367789 (263h54m54.316635419s) [noise=true , address=45.91.116.11:4147 , piece_id=23JASHPHO3VFWMEKGIT63COZGTWK3I3EEEKGRS7GX7SIFKIURILQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:16.506635739 (263h54m57.237903369s) [noise=true , address=77.90.31.16:4763 , piece_id=2V5IWUJHLF3YH3ZBQJFNH2JCGZ54SQASHQ5FLLGHEQOOSISFKWJQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:22.880177884 (263h55m3.611445533s) [noise=true , address=77.90.31.17:4783 , piece_id=RYTX2QLEKTYQTOR3NSZIPY2TWTBL2ZUKSJV5ORBKH5R2ZPXUKBNQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: <nil> Aug 19 17:21:30.678773057 (263h55m11.410040716s) [noise=true , address=45.91.116.7:4032 , piece_id=M2BXIGAGRRSUZ7EGWFA2IAFQV7ZRUZCH5NBWS4PYFACOGWN6COZQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:14fc8QANkMVxxrHcXTWi1yyDhE2gz8BJNwRuvJNfX56TyQCWYZ, address:45.91.116.7:4032): protocol: rpc: dial tcp 45.91.116.7:4032: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678847767 (263h55m11.410115416s) [noise=true , address=45.91.116.12:4238 , piece_id=6NKCMPVSRFRDDX3TU3ET2KOLQ2DCU2OVNZI5U6ZLUWP6JTF3PCWQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1yd47We7k2qgmcAVz5DhwZQMfnX71BrmLmtBoYyLn7D8t1psh2, address:45.91.116.12:4238): protocol: read tcp 5.161.89.252:32834->45.91.116.12:4238: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678926845 (263h55m11.410194495s) [noise=true , address=77.90.31.21:4890 , piece_id=5CIXXVAF25OX7BV7JI6QQ3UZ6JE62XC2GBZOYUMOQQXT3HUD5JOQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1iE3mDnEhGDsMBtmUXC4EwvSrpyHLoybzVXkC9eoFuYEMHeBrd, address:77.90.31.21:4890): protocol: rpc: dial tcp 77.90.31.21:4890: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678951581 (263h55m11.410219241s) [noise=true , address=77.90.31.18:4829 , piece_id=Y43CQMZRAZPDXAXNJJHFDX3C3LEKTFBBLKNRGMVWM7WUM34TVLHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1548pzNGffaMonic7dMK5VtZhLHtae3iffd9wjfjyTCp4XJ7Lo, address:77.90.31.18:4829): protocol: rpc: dial tcp 77.90.31.18:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.678966439 (263h55m11.410234099s) [noise=true , address=77.90.31.14:4695 , piece_id=I3HQFGDDWHDKIGJELHQXZ6MZQYCZ5TUDG4RK4M2OLJBNAS7TNWHQ, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:12q2kob5oYxuxwUDh4vjNDBpW4c6LndrjHbsJ4VRDiBsLxjgLdy, address:77.90.31.14:4695): protocol: write tcp 5.161.89.252:42466->77.90.31.14:4695: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679024038 (263h55m11.410291688s) [noise=true , address=45.91.116.10:4107 , piece_id=5Q5KSIA3G2LXQGXCAJRW2S7SKLOIQQYZ4EHMB3WZEB7UHA3F25WA, seg_pos={0 0}, upload=274457]: Done uploading piece. err: ecclient: upload failed (node:1U16aJ2LZPaRGyRizyyCbMJjUgtuDcaHaMsrQFbXHuyVGRc1w7, address:45.91.116.10:4107): protocol: read tcp 5.161.89.252:39718->45.91.116.10:4107: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:21:30.679071326 (263h55m11.410338977s) [seg_pos={0 0}, upload=274457]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Aug 19 17:21:30.694027780 (263h55m11.425295431s) [upload=274457]: Done uploading segments... Aug 19 17:21:30.694046335 (263h55m11.425313986s) [upload=274457]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration:* 1m0.017341115s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [..........XX..............................................X.] duration: 498.261977ms FAIL (2025-08-19 17:21:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 498.277207ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 4.260051598s FAIL (2025-08-19 17:21:24 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 4.260081185s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......X....X..............................................X.] duration: 3.394602438s FAIL (2025-08-19 17:21:12 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== Aug 19 17:20:25.233844508 (263h54m5.965112158s) [upload=274454]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:20:25.233849798 (263h54m5.965117448s) [upload=274454]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:25.340764042 (263h54m6.072031652s) [seg_pos={0 0}, upload=274454]: Begin upload segment... Aug 19 17:20:25.340793026 (263h54m6.072060646s) [seg_pos={0 0}, upload=274454]: Done begin upload segment. Aug 19 17:20:25.340821350 (263h54m6.072089041s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.340914916 (263h54m6.072182546s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341136623 (263h54m6.072404243s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341247761 (263h54m6.072515381s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341347639 (263h54m6.072615258s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341528018 (263h54m6.072795638s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341535622 (263h54m6.072803242s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341763621 (263h54m6.073031241s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341869960 (263h54m6.07313759s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342030262 (263h54m6.073297892s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342163261 (263h54m6.073430892s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342283257 (263h54m6.073550887s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342440372 (263h54m6.073708002s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342509302 (263h54m6.073776921s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342679041 (263h54m6.073946671s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.343286362 (263h54m6.074553981s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.429300735 (263h54m6.160568365s) [upload=274454]: Next returned nil segment Aug 19 17:20:25.429315603 (263h54m6.160583233s) [upload=274454]: Waiting for error group managing segments... Aug 19 17:20:25.442554419 (263h54m6.173822049s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.442663975 (263h54m6.173931605s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.443726752 (263h54m6.174994382s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.445744032 (263h54m6.177011652s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.459719312 (263h54m6.190986932s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.461728297 (263h54m6.192995927s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.467426444 (263h54m6.198694074s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:55.218201378 (263h54m35.949469008s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218314731 (263h54m35.949582361s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 5.161.89.252:23356->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218363783 (263h54m35.949631413s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: rpc: dial tcp 45.91.116.12:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218371297 (263h54m35.949638927s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218492334 (263h54m35.949759954s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218527600 (263h54m35.949795221s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218553008 (263h54m35.949820628s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 5.161.89.252:58664->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218563387 (263h54m35.949831008s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 5.161.89.252:21248->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218578345 (263h54m35.949845966s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 5.161.89.252:57908->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218608202 (263h54m35.949875823s) [seg_pos={0 0}, upload=274454]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 19 17:20:55.232496830 (263h54m35.96376445s) [upload=274454]: Done uploading segments... Aug 19 17:20:55.232513381 (263h54m35.963781001s) [upload=274454]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 3.39462432s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X...........X....] duration: 180.566459ms FAIL (2025-08-19 17:20:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 180.594442ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......X....X..............................................X] duration: 30.015449654s FAIL (2025-08-19 17:20:55 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== Aug 19 17:20:25.233844508 (263h54m5.965112158s) [upload=274454]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:20:25.233849798 (263h54m5.965117448s) [upload=274454]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:25.340764042 (263h54m6.072031652s) [seg_pos={0 0}, upload=274454]: Begin upload segment... Aug 19 17:20:25.340793026 (263h54m6.072060646s) [seg_pos={0 0}, upload=274454]: Done begin upload segment. Aug 19 17:20:25.340821350 (263h54m6.072089041s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.340914916 (263h54m6.072182546s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341136623 (263h54m6.072404243s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341247761 (263h54m6.072515381s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341347639 (263h54m6.072615258s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341528018 (263h54m6.072795638s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341535622 (263h54m6.072803242s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341763621 (263h54m6.073031241s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.341869960 (263h54m6.07313759s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342030262 (263h54m6.073297892s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342163261 (263h54m6.073430892s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342283257 (263h54m6.073550887s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342440372 (263h54m6.073708002s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342509302 (263h54m6.073776921s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.342679041 (263h54m6.073946671s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.343286362 (263h54m6.074553981s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Uploading piece... Aug 19 17:20:25.429300735 (263h54m6.160568365s) [upload=274454]: Next returned nil segment Aug 19 17:20:25.429315603 (263h54m6.160583233s) [upload=274454]: Waiting for error group managing segments... Aug 19 17:20:25.442554419 (263h54m6.173822049s) [noise=true , address=104.36.82.9:4006 , piece_id=VLEKROVJK6MQIS6PXQIAGUGNRX6QYHX3X6KOZVL53ZOBE2IQHMTQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.442663975 (263h54m6.173931605s) [noise=true , address=104.36.82.7:4909 , piece_id=KHVU4D2LKLDW5MXA4RIM4KFZ6YM3ITWOEKWLAUX6Y4BVYMDZX3OA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.443726752 (263h54m6.174994382s) [noise=true , address=104.36.82.10:4077 , piece_id=NY6LUCWWEC7BS67HEP4RJH4WD3BCIJ5TKBPFDJ7BRZG6N7QALB5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.445744032 (263h54m6.177011652s) [noise=true , address=104.36.82.3:4767 , piece_id=HTU4EJVFTXXYZNZU3Y3FLNMUQSG5L2W34GAID2HPPKEXXXIV7CMQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.459719312 (263h54m6.190986932s) [noise=true , address=104.36.82.4:4802 , piece_id=5A6RJXXMSSR6HX5XCALIUFMHWBF2S5VW3CPYTCH6QIC6V4ZHHKYQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.461728297 (263h54m6.192995927s) [noise=true , address=104.36.82.8:4972 , piece_id=WMD5UIQ6WJX6CTP4PQALMOOJCGKEUWAH43Q5GRROCMBBKXFBT5XA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:25.467426444 (263h54m6.198694074s) [noise=true , address=104.36.82.15:4258 , piece_id=HOMFQSXOYA64GXGZXCCHJT5FXAZRXM2YFIGNOOBGHQICZNMKR7PQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: <nil> Aug 19 17:20:55.218201378 (263h54m35.949469008s) [noise=true , address=77.90.31.21:4877 , piece_id=PLGHYWLOELO7HGOMQWKQBXBBFRK2P3BQORYH6CWO7ISIOV5BORKQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218314731 (263h54m35.949582361s) [noise=true , address=45.91.116.4:4938 , piece_id=GESYQXAGUPC7RWYE6BF3XPJFOT56JSXWKVDJLJDNLSX4CO7VOV5Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: read tcp 5.161.89.252:23356->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218363783 (263h54m35.949631413s) [noise=true , address=45.91.116.12:4227 , piece_id=RESLGK5RYOC7RVG32S5P2EIC44YO37YRLVBXDCGPIX6DH5ZY2D2A, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH, address:45.91.116.12:4227): protocol: rpc: dial tcp 45.91.116.12:4227: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218371297 (263h54m35.949638927s) [noise=true , address=77.90.31.16:4735 , piece_id=YGWILVSWNMZKOENBQOTCGVWEKFFNFR3BOSKC4GFODPZG7OQXTKGA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218492334 (263h54m35.949759954s) [noise=true , address=77.90.31.12:4592 , piece_id=NTXVWMHQPDHOZ2F7PJY5IBBLHY3SY4DP3RCUDZUNG62PGAROLNBQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218527600 (263h54m35.949795221s) [noise=true , address=77.90.31.16:4733 , piece_id=KF66NIB6JSTTPXIPFZAQPWPWOFOEDQAHRWZIUXL44HKLEIC7PNVA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf, address:77.90.31.16:4733): protocol: rpc: dial tcp 77.90.31.16:4733: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218553008 (263h54m35.949820628s) [noise=true , address=77.90.31.17:4772 , piece_id=ZSIHHFZXYM5UUIAHI5GKKO4I4LGVD642J5P7KTJ2ZF2RHNNHA2DQ, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 5.161.89.252:58664->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218563387 (263h54m35.949831008s) [noise=true , address=45.91.116.5:4297 , piece_id=IONJDOX7TQ72MKOE7GAS3OFQBGEO55AS33MGA6KIWSIKCCDT6WEA, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 5.161.89.252:21248->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218578345 (263h54m35.949845966s) [noise=true , address=45.91.116.12:4226 , piece_id=CFH5C72NRUTT4JEPQ22ZWJJHS4NXL4EFZ7EN7OXX2BREMUYBWC7Q, seg_pos={0 0}, upload=274454]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 5.161.89.252:57908->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 17:20:55.218608202 (263h54m35.949875823s) [seg_pos={0 0}, upload=274454]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Aug 19 17:20:55.232496830 (263h54m35.96376445s) [upload=274454]: Done uploading segments... Aug 19 17:20:55.232513381 (263h54m35.963781001s) [upload=274454]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.015471055s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...........XX..............................................X] duration: 30.001631052s FAIL (2025-08-19 17:20:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001649335s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................X....................................X] duration: 17.631951ms FAIL (2025-08-19 17:20:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:20:47.692563605 (263h54m28.423831256s) [upload=274464]: Starting upload Aug 19 17:20:47.692640950 (263h54m28.42390861s) [upload=274464]: Done starting upload Aug 19 17:20:47.692773529 (263h54m28.424041199s) [upload=274464]: Uploading segments... Aug 19 17:20:47.693054106 (263h54m28.424321756s) [upload=274464]: Got next segment. Inline: false Aug 19 17:20:47.693104080 (263h54m28.42437173s) [upload=274464]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:20:47.693110282 (263h54m28.424377932s) [upload=274464]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 17:20:47.696037212 (263h54m28.427304862s) [upload=274464]: Next segment err: metaclient: Too Many Requests Aug 19 17:20:47.696062039 (263h54m28.427329688s) [upload=274464]: Done uploading segments... Aug 19 17:20:47.696076836 (263h54m28.427344487s) [upload=274464]: Upload finished. err: metaclient: Too Many Requests Duration: 17.648082ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 1.47123493s FAIL (2025-08-19 17:20:07 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.471254256s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 1.816396548s FAIL (2025-08-19 17:20:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.816416695s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 458.873244ms FAIL (2025-08-19 17:19:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 16:54:39.768694265 (263h28m20.499961914s) [upload=273997]: Starting upload Aug 19 16:54:39.768758044 (263h28m20.500025694s) [upload=273997]: Done starting upload Aug 19 16:54:39.768833737 (263h28m20.500101387s) [upload=273997]: Uploading segments... Aug 19 16:54:39.769226274 (263h28m20.500493924s) [upload=273997]: Got next segment. Inline: false Aug 19 16:54:39.769329438 (263h28m20.500597089s) [upload=273997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 16:54:39.769336001 (263h28m20.500603651s) [upload=273997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 16:54:39.892373182 (263h28m20.623640832s) [seg_pos={0 0}, upload=273997]: Begin upload segment... Aug 19 16:54:39.892431482 (263h28m20.623699132s) [seg_pos={0 0}, upload=273997]: Done begin upload segment. Aug 19 16:54:39.892461147 (263h28m20.623728787s) [noise=true , address=104.36.82.23:4458 , piece_id=JRGAVETEMJJF3DNXAUDGVNCKZBW574VHBA3HMSQERYFBTXFVRAJQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.892496664 (263h28m20.623764314s) [noise=true , address=77.90.31.19:4860 , piece_id=H4QRMESKHG2VBNW6IPZRTTXMKCG5KT2EUBB3XW4SNQL5IWV7V4XQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.892931502 (263h28m20.624199151s) [noise=true , address=45.91.116.11:4145 , piece_id=HIPVV7SVTNCJ3RTPT7GZOL7FKY4BYEOIBACHGU426RTAG7GXH6MA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893245521 (263h28m20.624513162s) [noise=true , address=104.36.82.34:4562 , piece_id=A4ZF6PYWLA3R3TSNWSEMWXEYDCJJXYKEXZ7XCTR32G34YLLBPVUA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893286448 (263h28m20.624554098s) [noise=true , address=79.127.219.51:4200 , piece_id=OWKU7IP27IDNW7CIFG6CBPVDZRBR72OCQ7F6XX6EMUWWEEM5KTFA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893651074 (263h28m20.624918724s) [noise=true , address=79.127.201.215:4352 , piece_id=DXLVEBP2YYIUME2KXIUHOCH33FH6P4MEJD7EOP6BCNEJLJ37Y3RA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893836894 (263h28m20.625104543s) [noise=true , address=104.36.82.60:4357 , piece_id=DHMHWCJC4FXOXN2MJ2KZE5M3NEAUGZBM7UIOWLOB77UFWBWEIHCA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894009988 (263h28m20.625277639s) [noise=true , address=77.90.31.11:4579 , piece_id=E72566DABDPJRFHOB2BUUUMRM4I23RO2DLIMYPRILMAJDBX54XDA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894377059 (263h28m20.625644699s) [noise=true , address=77.90.31.17:4769 , piece_id=DVRHDX7IG4BLA3YABZAMINADGDYYQHCL5PG7RHQY5K5WC2RQ7BPQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894431541 (263h28m20.625699191s) [noise=true , address=77.90.31.19:4858 , piece_id=H2FLBEJW5FVFFY7Y7MI3KTQR36W5NAAVWZ3XJ4DQFP27DPH2DSVQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894732286 (263h28m20.625999926s) [noise=true , address=104.36.82.36:4627 , piece_id=TZGWDULYRO6X7IOIMD5NXVKA67MBD3PUXODS5UDCYIE2FFDP4DMQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.895057227 (263h28m20.626324877s) [noise=true , address=77.90.31.12:4619 , piece_id=G5CSYE7LDKSN2NDWGGLG232TC6HIFMD5ZIJOH5IJBPP5P4RHTVDA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.895256040 (263h28m20.626523701s) [noise=true , address=45.91.116.14:4206 , piece_id=AI3EDHQQH6TK7EELZMDQ3CH7FSFS5BWMEA3O2RUNF7OPVSGBDCWA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.895379082 (263h28m20.626646722s) [noise=true , address=45.91.116.10:4084 , piece_id=L5STTBHOA4FX5GPTHWWPC5RWWSVGIJE7VWDN5WRCDP6IS2GBPCJQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.896068288 (263h28m20.627335948s) [noise=true , address=104.36.82.11:4128 , piece_id=GOLKDJAL3T5V3KM4UDLE7PO3GH4SJCWUZMBTG4QE6LAXDR3KLPDA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.896754376 (263h28m20.628022026s) [noise=true , address=45.91.116.11:4128 , piece_id=YLO2NLMW7S2RETXLR4KCBTWI4RHY6I4CYKCDXV7MT2Z2TXUZECKQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.897242674 (263h28m20.628510314s) [noise=true , address=79.127.219.47:4048 , piece_id=QTLYJYLL4D5PDV46IQBVUVMNOXKPGQCNRZ2SGJPWRL5PE5WDEXLA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.897649399 (263h28m20.628917049s) [noise=true , address=104.36.82.49:4002 , piece_id=BON56TN53YTZ42H3UGSCYPBXH7T3YYIHLVNURECRGVGCSH2XF6CQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.996733958 (263h28m20.728001578s) [upload=273997]: Next returned nil segment Aug 19 16:54:39.996744999 (263h28m20.728012619s) [upload=273997]: Waiting for error group managing segments... Aug 19 16:54:40.006905447 (263h28m20.738173067s) [noise=true , address=45.91.116.14:4206 , piece_id=AI3EDHQQH6TK7EELZMDQ3CH7FSFS5BWMEA3O2RUNF7OPVSGBDCWA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.010650045 (263h28m20.741917695s) [noise=true , address=104.36.82.49:4002 , piece_id=BON56TN53YTZ42H3UGSCYPBXH7T3YYIHLVNURECRGVGCSH2XF6CQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.012387941 (263h28m20.743655582s) [noise=true , address=104.36.82.11:4128 , piece_id=GOLKDJAL3T5V3KM4UDLE7PO3GH4SJCWUZMBTG4QE6LAXDR3KLPDA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.012600531 (263h28m20.743868191s) [noise=true , address=45.91.116.11:4145 , piece_id=HIPVV7SVTNCJ3RTPT7GZOL7FKY4BYEOIBACHGU426RTAG7GXH6MA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.013127341 (263h28m20.744394981s) [noise=true , address=45.91.116.11:4128 , piece_id=YLO2NLMW7S2RETXLR4KCBTWI4RHY6I4CYKCDXV7MT2Z2TXUZECKQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.015725054 (263h28m20.746992704s) [noise=true , address=79.127.219.51:4200 , piece_id=OWKU7IP27IDNW7CIFG6CBPVDZRBR72OCQ7F6XX6EMUWWEEM5KTFA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.017748017 (263h28m20.749015658s) [noise=true , address=45.91.116.10:4084 , piece_id=L5STTBHOA4FX5GPTHWWPC5RWWSVGIJE7VWDN5WRCDP6IS2GBPCJQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.022367990 (263h28m20.75363564s) [noise=true , address=79.127.219.47:4048 , piece_id=QTLYJYLL4D5PDV46IQBVUVMNOXKPGQCNRZ2SGJPWRL5PE5WDEXLA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.023459282 (263h28m20.754726912s) [noise=true , address=104.36.82.60:4357 , piece_id=DHMHWCJC4FXOXN2MJ2KZE5M3NEAUGZBM7UIOWLOB77UFWBWEIHCA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.023905380 (263h28m20.755173031s) [noise=true , address=77.90.31.12:4619 , piece_id=G5CSYE7LDKSN2NDWGGLG232TC6HIFMD5ZIJOH5IJBPP5P4RHTVDA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.028678080 (263h28m20.759945721s) [noise=true , address=104.36.82.34:4562 , piece_id=A4ZF6PYWLA3R3TSNWSEMWXEYDCJJXYKEXZ7XCTR32G34YLLBPVUA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.035459938 (263h28m20.766727568s) [noise=true , address=77.90.31.17:4769 , piece_id=DVRHDX7IG4BLA3YABZAMINADGDYYQHCL5PG7RHQY5K5WC2RQ7BPQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.038353317 (263h28m20.769620966s) [noise=true , address=77.90.31.19:4860 , piece_id=H4QRMESKHG2VBNW6IPZRTTXMKCG5KT2EUBB3XW4SNQL5IWV7V4XQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.045552748 (263h28m20.776820388s) [noise=true , address=104.36.82.36:4627 , piece_id=TZGWDULYRO6X7IOIMD5NXVKA67MBD3PUXODS5UDCYIE2FFDP4DMQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.045575121 (263h28m20.776842761s) [seg_pos={0 0}, upload=273997]: Segment reached optimal threshold of 14 pieces. Aug 19 16:54:40.045725383 (263h28m20.776993023s) [noise=true , address=104.36.82.23:4458 , piece_id=JRGAVETEMJJF3DNXAUDGVNCKZBW574VHBA3HMSQERYFBTXFVRAJQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TMPdLfCbmgi3GqDDg6AUvTCFPzbCCgwTAn9JJBrYJRnbkKtpb): protocol: expected piece hash; context canceled; EOF Aug 19 16:54:40.045862330 (263h28m20.777129961s) [noise=true , address=77.90.31.11:4579 , piece_id=E72566DABDPJRFHOB2BUUUMRM4I23RO2DLIMYPRILMAJDBX54XDA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dRcuBLuT2ybWGzdkp4vyo7e37qrZk6SBpWWHty5WKRoVHp1p): protocol: expected piece hash; context canceled; EOF Aug 19 16:54:40.045955475 (263h28m20.777223116s) [noise=true , address=79.127.201.215:4352 , piece_id=DXLVEBP2YYIUME2KXIUHOCH33FH6P4MEJD7EOP6BCNEJLJ37Y3RA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: write tcp 5.161.89.252:60156->79.127.201.215:4352: use of closed network connection; write tcp 5.161.89.252:60156->79.127.201.215:4352: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:60156->79.127.201.215:4352: use of closed network connection Aug 19 16:54:40.045990340 (263h28m20.777257991s) [noise=true , address=77.90.31.19:4858 , piece_id=H2FLBEJW5FVFFY7Y7MI3KTQR36W5NAAVWZ3XJ4DQFP27DPH2DSVQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F7WDweCEEWG1kxAAQpJPcigEHsrjuqpuGhGTzNPPYTsXEX43c): protocol: expected piece hash; context canceled; EOF Aug 19 16:54:40.046041047 (263h28m20.777308707s) [seg_pos={0 0}, upload=273997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 16:54:40.046213360 (263h28m20.77748101s) [upload=273997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 16:54:40.046222577 (263h28m20.777490227s) [upload=273997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 16:54:40.507200222 (263h28m21.238467872s) [upload=273997]: Done uploading segments... Aug 19 16:54:40.507219628 (263h28m21.238487279s) [upload=273997]: Upload finished. err: <nil> Duration: 458.899133ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................................X...........X...] duration: 441.894883ms FAIL (2025-08-19 17:19:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 441.920081ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X....X..............................................] duration: 426.553085ms FAIL (2025-08-19 17:19:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== HUB2ATULVRJUG2X4NKN4FEZX76ZX6UMSKQJCMG44M6GQQ, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.440439150 (263h22m29.171706769s) [noise=true , address=45.91.116.4:4939 , piece_id=ZM5HC2WHMWXI5PKNIBWY7CQFTTLKYGDS5JTCDW7VXVFHSUH5VFOQ, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.440660145 (263h22m29.171927765s) [noise=true , address=104.36.82.3:4765 , piece_id=FICBIS72PKYKS6AOUULD7HW3YHBWKFD47XDRNWXQ2SUCHEAPCFXQ, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.440690993 (263h22m29.171958613s) [noise=true , address=77.90.31.17:4770 , piece_id=Y6E4P2UQ2UWLUEXFZBTYOP5FEU3BBVOU4ROP7BE23WWJCGWGGXPQ, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.440892572 (263h22m29.172160192s) [noise=true , address=104.36.82.4:4801 , piece_id=U7MZUQQMFM62TRK2FKUM4EXWCFPF37HPRXKUDSNJRGIFD2I7CRKA, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.441093340 (263h22m29.17236097s) [noise=true , address=45.91.116.10:4083 , piece_id=DCGONYE5WFQMJY5OIJA2GQRWCE5DWRYHASKZL6MQ3E7D3NN4FW4Q, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.441169954 (263h22m29.172437574s) [noise=true , address=45.91.116.4:4938 , piece_id=ZNNQF36YF5PYBM4UVIBXW3J3JDQA5KSOR4QX7IFZFJ44PFO5A47A, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.441405507 (263h22m29.172673137s) [noise=true , address=45.91.116.6:4975 , piece_id=DJQ6P2UCKB3LQSB7AXSD4OPYJZBRKQLT4Q4MDKMRBEFI7WBBJQ7Q, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.441449650 (263h22m29.17271728s) [noise=true , address=45.91.116.13:4153 , piece_id=3U2Q2V4STGSAQMCBABLU2MTFFJHE4SVM6KDPIB65BCJO2YKHHQVQ, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.441634066 (263h22m29.172901696s) [noise=true , address=45.91.116.11:4117 , piece_id=KMZCIAUNTAPDMVJPET664DJPPOPT246HG7CPGLFSMSU6PYJPBG7Q, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.441871643 (263h22m29.173139263s) [noise=true , address=104.36.82.4:4803 , piece_id=WZUVKIJLQQ7KHHWTJOCANFAQUR6K6SZIV5D4ERBI5C6XWWXOB4SA, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.441878365 (263h22m29.173145995s) [noise=true , address=45.91.116.7:4009 , piece_id=4B2VSIA2WUWM63H2ECCN2LPKK4Y4P4QJAZSKU3N3OHS5VAQ4B64Q, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.442236228 (263h22m29.173503848s) [noise=true , address=77.90.31.13:4627 , piece_id=C76Y3OG643BPEB4KDBB77CSXZOYKKZLUJT2UUNZBRM4SSJBAXC2A, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.444182237 (263h22m29.175449867s) [noise=true , address=104.36.82.13:4186 , piece_id=ZESX7QFJI6TEZW4WLDEOR6FPWA4NFY2MMSCHBMY7TATXRZRUFNDA, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.445002428 (263h22m29.176270058s) [noise=true , address=77.90.31.15:4697 , piece_id=JD3LN3TJRZBVHDMCTRQDFLJUKWLZ2IMCC3YO7QNAXDRKE5F53OYA, seg_pos={0 0}, upload=273893]: Uploading piece... Aug 19 16:48:48.521537208 (263h22m29.252804838s) [upload=273893]: Next returned nil segment Aug 19 16:48:48.521548239 (263h22m29.252815869s) [upload=273893]: Waiting for error group managing segments... Aug 19 16:48:48.541405319 (263h22m29.272672938s) [noise=true , address=104.36.82.4:4801 , piece_id=U7MZUQQMFM62TRK2FKUM4EXWCFPF37HPRXKUDSNJRGIFD2I7CRKA, seg_pos={0 0}, upload=273893]: Done uploading piece. err: <nil> Aug 19 16:48:48.545658234 (263h22m29.276925864s) [noise=true , address=104.36.82.4:4803 , piece_id=WZUVKIJLQQ7KHHWTJOCANFAQUR6K6SZIV5D4ERBI5C6XWWXOB4SA, seg_pos={0 0}, upload=273893]: Done uploading piece. err: <nil> Aug 19 16:48:48.557936526 (263h22m29.289204156s) [noise=true , address=104.36.82.3:4765 , piece_id=FICBIS72PKYKS6AOUULD7HW3YHBWKFD47XDRNWXQ2SUCHEAPCFXQ, seg_pos={0 0}, upload=273893]: Done uploading piece. err: <nil> Aug 19 16:48:48.564988173 (263h22m29.296255792s) [noise=true , address=104.36.82.13:4186 , piece_id=ZESX7QFJI6TEZW4WLDEOR6FPWA4NFY2MMSCHBMY7TATXRZRUFNDA, seg_pos={0 0}, upload=273893]: Done uploading piece. err: <nil> Aug 19 16:49:18.351552468 (263h22m59.082820118s) [noise=true , address=45.91.116.4:4938 , piece_id=ZNNQF36YF5PYBM4UVIBXW3J3JDQA5KSOR4QX7IFZFJ44PFO5A47A, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.351637016 (263h22m59.082904667s) [noise=true , address=77.90.31.15:4697 , piece_id=JD3LN3TJRZBVHDMCTRQDFLJUKWLZ2IMCC3YO7QNAXDRKE5F53OYA, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: rpc: dial tcp 77.90.31.15:4697: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.351690206 (263h22m59.082957866s) [noise=true , address=77.90.31.19:4842 , piece_id=JKRQJGZHUB2ATULVRJUG2X4NKN4FEZX76ZX6UMSKQJCMG44M6GQQ, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.351763084 (263h22m59.083030734s) [noise=true , address=45.91.116.7:4009 , piece_id=4B2VSIA2WUWM63H2ECCN2LPKK4Y4P4QJAZSKU3N3OHS5VAQ4B64Q, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.351882037 (263h22m59.083149697s) [noise=true , address=45.91.116.6:4975 , piece_id=DJQ6P2UCKB3LQSB7AXSD4OPYJZBRKQLT4Q4MDKMRBEFI7WBBJQ7Q, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: read tcp 5.161.89.252:48890->45.91.116.6:4975: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.351982256 (263h22m59.083249916s) [noise=true , address=45.91.116.4:4939 , piece_id=ZM5HC2WHMWXI5PKNIBWY7CQFTTLKYGDS5JTCDW7VXVFHSUH5VFOQ, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.352020067 (263h22m59.083287717s) [noise=true , address=45.91.116.11:4117 , piece_id=KMZCIAUNTAPDMVJPET664DJPPOPT246HG7CPGLFSMSU6PYJPBG7Q, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: rpc: dial tcp 45.91.116.11:4117: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.352157445 (263h22m59.083425104s) [noise=true , address=45.91.116.13:4153 , piece_id=3U2Q2V4STGSAQMCBABLU2MTFFJHE4SVM6KDPIB65BCJO2YKHHQVQ, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 5.161.89.252:14982->45.91.116.13:4153: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.352213380 (263h22m59.083481031s) [noise=true , address=45.91.116.10:4083 , piece_id=DCGONYE5WFQMJY5OIJA2GQRWCE5DWRYHASKZL6MQ3E7D3NN4FW4Q, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 5.161.89.252:12880->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.352259367 (263h22m59.083527016s) [noise=true , address=77.90.31.17:4770 , piece_id=Y6E4P2UQ2UWLUEXFZBTYOP5FEU3BBVOU4ROP7BE23WWJCGWGGXPQ, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R, address:77.90.31.17:4770): protocol: read tcp 5.161.89.252:20282->77.90.31.17:4770: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.352279484 (263h22m59.083547134s) [noise=true , address=77.90.31.13:4627 , piece_id=C76Y3OG643BPEB4KDBB77CSXZOYKKZLUJT2UUNZBRM4SSJBAXC2A, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 5.161.89.252:29268->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.352328276 (263h22m59.083595926s) [noise=true , address=77.90.31.14:4661 , piece_id=4HJKLDK4ATJU4IWPD2F45TCJSYXNLO2YCC2RTMK3CGARSRJKSBWQ, seg_pos={0 0}, upload=273893]: Done uploading piece. err: ecclient: upload failed (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue, address:77.90.31.14:4661): protocol: read tcp 5.161.89.252:26624->77.90.31.14:4661: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 19 16:49:18.352362249 (263h22m59.0836299s) [seg_pos={0 0}, upload=273893]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Aug 19 16:49:18.364568045 (263h22m59.095835696s) [upload=273893]: Done uploading segments... Aug 19 16:49:18.364591099 (263h22m59.095858749s) [upload=273893]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 426.584875ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 394.57654ms FAIL (2025-08-19 17:19:34 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 16:54:39.768694265 (263h28m20.499961914s) [upload=273997]: Starting upload Aug 19 16:54:39.768758044 (263h28m20.500025694s) [upload=273997]: Done starting upload Aug 19 16:54:39.768833737 (263h28m20.500101387s) [upload=273997]: Uploading segments... Aug 19 16:54:39.769226274 (263h28m20.500493924s) [upload=273997]: Got next segment. Inline: false Aug 19 16:54:39.769329438 (263h28m20.500597089s) [upload=273997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 16:54:39.769336001 (263h28m20.500603651s) [upload=273997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 16:54:39.892373182 (263h28m20.623640832s) [seg_pos={0 0}, upload=273997]: Begin upload segment... Aug 19 16:54:39.892431482 (263h28m20.623699132s) [seg_pos={0 0}, upload=273997]: Done begin upload segment. Aug 19 16:54:39.892461147 (263h28m20.623728787s) [noise=true , address=104.36.82.23:4458 , piece_id=JRGAVETEMJJF3DNXAUDGVNCKZBW574VHBA3HMSQERYFBTXFVRAJQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.892496664 (263h28m20.623764314s) [noise=true , address=77.90.31.19:4860 , piece_id=H4QRMESKHG2VBNW6IPZRTTXMKCG5KT2EUBB3XW4SNQL5IWV7V4XQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.892931502 (263h28m20.624199151s) [noise=true , address=45.91.116.11:4145 , piece_id=HIPVV7SVTNCJ3RTPT7GZOL7FKY4BYEOIBACHGU426RTAG7GXH6MA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893245521 (263h28m20.624513162s) [noise=true , address=104.36.82.34:4562 , piece_id=A4ZF6PYWLA3R3TSNWSEMWXEYDCJJXYKEXZ7XCTR32G34YLLBPVUA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893286448 (263h28m20.624554098s) [noise=true , address=79.127.219.51:4200 , piece_id=OWKU7IP27IDNW7CIFG6CBPVDZRBR72OCQ7F6XX6EMUWWEEM5KTFA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893651074 (263h28m20.624918724s) [noise=true , address=79.127.201.215:4352 , piece_id=DXLVEBP2YYIUME2KXIUHOCH33FH6P4MEJD7EOP6BCNEJLJ37Y3RA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893836894 (263h28m20.625104543s) [noise=true , address=104.36.82.60:4357 , piece_id=DHMHWCJC4FXOXN2MJ2KZE5M3NEAUGZBM7UIOWLOB77UFWBWEIHCA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894009988 (263h28m20.625277639s) [noise=true , address=77.90.31.11:4579 , piece_id=E72566DABDPJRFHOB2BUUUMRM4I23RO2DLIMYPRILMAJDBX54XDA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894377059 (263h28m20.625644699s) [noise=true , address=77.90.31.17:4769 , piece_id=DVRHDX7IG4BLA3YABZAMINADGDYYQHCL5PG7RHQY5K5WC2RQ7BPQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894431541 (263h28m20.625699191s) [noise=true , address=77.90.31.19:4858 , piece_id=H2FLBEJW5FVFFY7Y7MI3KTQR36W5NAAVWZ3XJ4DQFP27DPH2DSVQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894732286 (263h28m20.625999926s) [noise=true , address=104.36.82.36:4627 , piece_id=TZGWDULYRO6X7IOIMD5NXVKA67MBD3PUXODS5UDCYIE2FFDP4DMQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.895057227 (263h28m20.626324877s) [noise=true , address=77.90.31.12:4619 , piece_id=G5CSYE7LDKSN2NDWGGLG232TC6HIFMD5ZIJOH5IJBPP5P4RHTVDA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.895256040 (263h28m20.626523701s) [noise=true , address=45.91.116.14:4206 , piece_id=AI3EDHQQH6TK7EELZMDQ3CH7FSFS5BWMEA3O2RUNF7OPVSGBDCWA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.895379082 (263h28m20.626646722s) [noise=true , address=45.91.116.10:4084 , piece_id=L5STTBHOA4FX5GPTHWWPC5RWWSVGIJE7VWDN5WRCDP6IS2GBPCJQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.896068288 (263h28m20.627335948s) [noise=true , address=104.36.82.11:4128 , piece_id=GOLKDJAL3T5V3KM4UDLE7PO3GH4SJCWUZMBTG4QE6LAXDR3KLPDA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.896754376 (263h28m20.628022026s) [noise=true , address=45.91.116.11:4128 , piece_id=YLO2NLMW7S2RETXLR4KCBTWI4RHY6I4CYKCDXV7MT2Z2TXUZECKQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.897242674 (263h28m20.628510314s) [noise=true , address=79.127.219.47:4048 , piece_id=QTLYJYLL4D5PDV46IQBVUVMNOXKPGQCNRZ2SGJPWRL5PE5WDEXLA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.897649399 (263h28m20.628917049s) [noise=true , address=104.36.82.49:4002 , piece_id=BON56TN53YTZ42H3UGSCYPBXH7T3YYIHLVNURECRGVGCSH2XF6CQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.996733958 (263h28m20.728001578s) [upload=273997]: Next returned nil segment Aug 19 16:54:39.996744999 (263h28m20.728012619s) [upload=273997]: Waiting for error group managing segments... Aug 19 16:54:40.006905447 (263h28m20.738173067s) [noise=true , address=45.91.116.14:4206 , piece_id=AI3EDHQQH6TK7EELZMDQ3CH7FSFS5BWMEA3O2RUNF7OPVSGBDCWA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.010650045 (263h28m20.741917695s) [noise=true , address=104.36.82.49:4002 , piece_id=BON56TN53YTZ42H3UGSCYPBXH7T3YYIHLVNURECRGVGCSH2XF6CQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.012387941 (263h28m20.743655582s) [noise=true , address=104.36.82.11:4128 , piece_id=GOLKDJAL3T5V3KM4UDLE7PO3GH4SJCWUZMBTG4QE6LAXDR3KLPDA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.012600531 (263h28m20.743868191s) [noise=true , address=45.91.116.11:4145 , piece_id=HIPVV7SVTNCJ3RTPT7GZOL7FKY4BYEOIBACHGU426RTAG7GXH6MA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.013127341 (263h28m20.744394981s) [noise=true , address=45.91.116.11:4128 , piece_id=YLO2NLMW7S2RETXLR4KCBTWI4RHY6I4CYKCDXV7MT2Z2TXUZECKQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.015725054 (263h28m20.746992704s) [noise=true , address=79.127.219.51:4200 , piece_id=OWKU7IP27IDNW7CIFG6CBPVDZRBR72OCQ7F6XX6EMUWWEEM5KTFA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.017748017 (263h28m20.749015658s) [noise=true , address=45.91.116.10:4084 , piece_id=L5STTBHOA4FX5GPTHWWPC5RWWSVGIJE7VWDN5WRCDP6IS2GBPCJQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.022367990 (263h28m20.75363564s) [noise=true , address=79.127.219.47:4048 , piece_id=QTLYJYLL4D5PDV46IQBVUVMNOXKPGQCNRZ2SGJPWRL5PE5WDEXLA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.023459282 (263h28m20.754726912s) [noise=true , address=104.36.82.60:4357 , piece_id=DHMHWCJC4FXOXN2MJ2KZE5M3NEAUGZBM7UIOWLOB77UFWBWEIHCA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.023905380 (263h28m20.755173031s) [noise=true , address=77.90.31.12:4619 , piece_id=G5CSYE7LDKSN2NDWGGLG232TC6HIFMD5ZIJOH5IJBPP5P4RHTVDA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.028678080 (263h28m20.759945721s) [noise=true , address=104.36.82.34:4562 , piece_id=A4ZF6PYWLA3R3TSNWSEMWXEYDCJJXYKEXZ7XCTR32G34YLLBPVUA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.035459938 (263h28m20.766727568s) [noise=true , address=77.90.31.17:4769 , piece_id=DVRHDX7IG4BLA3YABZAMINADGDYYQHCL5PG7RHQY5K5WC2RQ7BPQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.038353317 (263h28m20.769620966s) [noise=true , address=77.90.31.19:4860 , piece_id=H4QRMESKHG2VBNW6IPZRTTXMKCG5KT2EUBB3XW4SNQL5IWV7V4XQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.045552748 (263h28m20.776820388s) [noise=true , address=104.36.82.36:4627 , piece_id=TZGWDULYRO6X7IOIMD5NXVKA67MBD3PUXODS5UDCYIE2FFDP4DMQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.045575121 (263h28m20.776842761s) [seg_pos={0 0}, upload=273997]: Segment reached optimal threshold of 14 pieces. Aug 19 16:54:40.045725383 (263h28m20.776993023s) [noise=true , address=104.36.82.23:4458 , piece_id=JRGAVETEMJJF3DNXAUDGVNCKZBW574VHBA3HMSQERYFBTXFVRAJQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TMPdLfCbmgi3GqDDg6AUvTCFPzbCCgwTAn9JJBrYJRnbkKtpb): protocol: expected piece hash; context canceled; EOF Aug 19 16:54:40.045862330 (263h28m20.777129961s) [noise=true , address=77.90.31.11:4579 , piece_id=E72566DABDPJRFHOB2BUUUMRM4I23RO2DLIMYPRILMAJDBX54XDA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dRcuBLuT2ybWGzdkp4vyo7e37qrZk6SBpWWHty5WKRoVHp1p): protocol: expected piece hash; context canceled; EOF Aug 19 16:54:40.045955475 (263h28m20.777223116s) [noise=true , address=79.127.201.215:4352 , piece_id=DXLVEBP2YYIUME2KXIUHOCH33FH6P4MEJD7EOP6BCNEJLJ37Y3RA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: write tcp 5.161.89.252:60156->79.127.201.215:4352: use of closed network connection; write tcp 5.161.89.252:60156->79.127.201.215:4352: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:60156->79.127.201.215:4352: use of closed network connection Aug 19 16:54:40.045990340 (263h28m20.777257991s) [noise=true , address=77.90.31.19:4858 , piece_id=H2FLBEJW5FVFFY7Y7MI3KTQR36W5NAAVWZ3XJ4DQFP27DPH2DSVQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F7WDweCEEWG1kxAAQpJPcigEHsrjuqpuGhGTzNPPYTsXEX43c): protocol: expected piece hash; context canceled; EOF Aug 19 16:54:40.046041047 (263h28m20.777308707s) [seg_pos={0 0}, upload=273997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 16:54:40.046213360 (263h28m20.77748101s) [upload=273997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 16:54:40.046222577 (263h28m20.777490227s) [upload=273997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 16:54:40.507200222 (263h28m21.238467872s) [upload=273997]: Done uploading segments... Aug 19 16:54:40.507219628 (263h28m21.238487279s) [upload=273997]: Upload finished. err: <nil> Duration: 394.592018ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.31906391s FAIL (2025-08-19 17:19:18 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 1.319085319s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 670.770806ms FAIL (2025-08-19 17:19:10 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 19 16:54:39.768694265 (263h28m20.499961914s) [upload=273997]: Starting upload Aug 19 16:54:39.768758044 (263h28m20.500025694s) [upload=273997]: Done starting upload Aug 19 16:54:39.768833737 (263h28m20.500101387s) [upload=273997]: Uploading segments... Aug 19 16:54:39.769226274 (263h28m20.500493924s) [upload=273997]: Got next segment. Inline: false Aug 19 16:54:39.769329438 (263h28m20.500597089s) [upload=273997]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 16:54:39.769336001 (263h28m20.500603651s) [upload=273997]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 19 16:54:39.892373182 (263h28m20.623640832s) [seg_pos={0 0}, upload=273997]: Begin upload segment... Aug 19 16:54:39.892431482 (263h28m20.623699132s) [seg_pos={0 0}, upload=273997]: Done begin upload segment. Aug 19 16:54:39.892461147 (263h28m20.623728787s) [noise=true , address=104.36.82.23:4458 , piece_id=JRGAVETEMJJF3DNXAUDGVNCKZBW574VHBA3HMSQERYFBTXFVRAJQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.892496664 (263h28m20.623764314s) [noise=true , address=77.90.31.19:4860 , piece_id=H4QRMESKHG2VBNW6IPZRTTXMKCG5KT2EUBB3XW4SNQL5IWV7V4XQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.892931502 (263h28m20.624199151s) [noise=true , address=45.91.116.11:4145 , piece_id=HIPVV7SVTNCJ3RTPT7GZOL7FKY4BYEOIBACHGU426RTAG7GXH6MA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893245521 (263h28m20.624513162s) [noise=true , address=104.36.82.34:4562 , piece_id=A4ZF6PYWLA3R3TSNWSEMWXEYDCJJXYKEXZ7XCTR32G34YLLBPVUA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893286448 (263h28m20.624554098s) [noise=true , address=79.127.219.51:4200 , piece_id=OWKU7IP27IDNW7CIFG6CBPVDZRBR72OCQ7F6XX6EMUWWEEM5KTFA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893651074 (263h28m20.624918724s) [noise=true , address=79.127.201.215:4352 , piece_id=DXLVEBP2YYIUME2KXIUHOCH33FH6P4MEJD7EOP6BCNEJLJ37Y3RA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.893836894 (263h28m20.625104543s) [noise=true , address=104.36.82.60:4357 , piece_id=DHMHWCJC4FXOXN2MJ2KZE5M3NEAUGZBM7UIOWLOB77UFWBWEIHCA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894009988 (263h28m20.625277639s) [noise=true , address=77.90.31.11:4579 , piece_id=E72566DABDPJRFHOB2BUUUMRM4I23RO2DLIMYPRILMAJDBX54XDA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894377059 (263h28m20.625644699s) [noise=true , address=77.90.31.17:4769 , piece_id=DVRHDX7IG4BLA3YABZAMINADGDYYQHCL5PG7RHQY5K5WC2RQ7BPQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894431541 (263h28m20.625699191s) [noise=true , address=77.90.31.19:4858 , piece_id=H2FLBEJW5FVFFY7Y7MI3KTQR36W5NAAVWZ3XJ4DQFP27DPH2DSVQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.894732286 (263h28m20.625999926s) [noise=true , address=104.36.82.36:4627 , piece_id=TZGWDULYRO6X7IOIMD5NXVKA67MBD3PUXODS5UDCYIE2FFDP4DMQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.895057227 (263h28m20.626324877s) [noise=true , address=77.90.31.12:4619 , piece_id=G5CSYE7LDKSN2NDWGGLG232TC6HIFMD5ZIJOH5IJBPP5P4RHTVDA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.895256040 (263h28m20.626523701s) [noise=true , address=45.91.116.14:4206 , piece_id=AI3EDHQQH6TK7EELZMDQ3CH7FSFS5BWMEA3O2RUNF7OPVSGBDCWA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.895379082 (263h28m20.626646722s) [noise=true , address=45.91.116.10:4084 , piece_id=L5STTBHOA4FX5GPTHWWPC5RWWSVGIJE7VWDN5WRCDP6IS2GBPCJQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.896068288 (263h28m20.627335948s) [noise=true , address=104.36.82.11:4128 , piece_id=GOLKDJAL3T5V3KM4UDLE7PO3GH4SJCWUZMBTG4QE6LAXDR3KLPDA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.896754376 (263h28m20.628022026s) [noise=true , address=45.91.116.11:4128 , piece_id=YLO2NLMW7S2RETXLR4KCBTWI4RHY6I4CYKCDXV7MT2Z2TXUZECKQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.897242674 (263h28m20.628510314s) [noise=true , address=79.127.219.47:4048 , piece_id=QTLYJYLL4D5PDV46IQBVUVMNOXKPGQCNRZ2SGJPWRL5PE5WDEXLA, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.897649399 (263h28m20.628917049s) [noise=true , address=104.36.82.49:4002 , piece_id=BON56TN53YTZ42H3UGSCYPBXH7T3YYIHLVNURECRGVGCSH2XF6CQ, seg_pos={0 0}, upload=273997]: Uploading piece... Aug 19 16:54:39.996733958 (263h28m20.728001578s) [upload=273997]: Next returned nil segment Aug 19 16:54:39.996744999 (263h28m20.728012619s) [upload=273997]: Waiting for error group managing segments... Aug 19 16:54:40.006905447 (263h28m20.738173067s) [noise=true , address=45.91.116.14:4206 , piece_id=AI3EDHQQH6TK7EELZMDQ3CH7FSFS5BWMEA3O2RUNF7OPVSGBDCWA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.010650045 (263h28m20.741917695s) [noise=true , address=104.36.82.49:4002 , piece_id=BON56TN53YTZ42H3UGSCYPBXH7T3YYIHLVNURECRGVGCSH2XF6CQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.012387941 (263h28m20.743655582s) [noise=true , address=104.36.82.11:4128 , piece_id=GOLKDJAL3T5V3KM4UDLE7PO3GH4SJCWUZMBTG4QE6LAXDR3KLPDA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.012600531 (263h28m20.743868191s) [noise=true , address=45.91.116.11:4145 , piece_id=HIPVV7SVTNCJ3RTPT7GZOL7FKY4BYEOIBACHGU426RTAG7GXH6MA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.013127341 (263h28m20.744394981s) [noise=true , address=45.91.116.11:4128 , piece_id=YLO2NLMW7S2RETXLR4KCBTWI4RHY6I4CYKCDXV7MT2Z2TXUZECKQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.015725054 (263h28m20.746992704s) [noise=true , address=79.127.219.51:4200 , piece_id=OWKU7IP27IDNW7CIFG6CBPVDZRBR72OCQ7F6XX6EMUWWEEM5KTFA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.017748017 (263h28m20.749015658s) [noise=true , address=45.91.116.10:4084 , piece_id=L5STTBHOA4FX5GPTHWWPC5RWWSVGIJE7VWDN5WRCDP6IS2GBPCJQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.022367990 (263h28m20.75363564s) [noise=true , address=79.127.219.47:4048 , piece_id=QTLYJYLL4D5PDV46IQBVUVMNOXKPGQCNRZ2SGJPWRL5PE5WDEXLA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.023459282 (263h28m20.754726912s) [noise=true , address=104.36.82.60:4357 , piece_id=DHMHWCJC4FXOXN2MJ2KZE5M3NEAUGZBM7UIOWLOB77UFWBWEIHCA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.023905380 (263h28m20.755173031s) [noise=true , address=77.90.31.12:4619 , piece_id=G5CSYE7LDKSN2NDWGGLG232TC6HIFMD5ZIJOH5IJBPP5P4RHTVDA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.028678080 (263h28m20.759945721s) [noise=true , address=104.36.82.34:4562 , piece_id=A4ZF6PYWLA3R3TSNWSEMWXEYDCJJXYKEXZ7XCTR32G34YLLBPVUA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.035459938 (263h28m20.766727568s) [noise=true , address=77.90.31.17:4769 , piece_id=DVRHDX7IG4BLA3YABZAMINADGDYYQHCL5PG7RHQY5K5WC2RQ7BPQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.038353317 (263h28m20.769620966s) [noise=true , address=77.90.31.19:4860 , piece_id=H4QRMESKHG2VBNW6IPZRTTXMKCG5KT2EUBB3XW4SNQL5IWV7V4XQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.045552748 (263h28m20.776820388s) [noise=true , address=104.36.82.36:4627 , piece_id=TZGWDULYRO6X7IOIMD5NXVKA67MBD3PUXODS5UDCYIE2FFDP4DMQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: <nil> Aug 19 16:54:40.045575121 (263h28m20.776842761s) [seg_pos={0 0}, upload=273997]: Segment reached optimal threshold of 14 pieces. Aug 19 16:54:40.045725383 (263h28m20.776993023s) [noise=true , address=104.36.82.23:4458 , piece_id=JRGAVETEMJJF3DNXAUDGVNCKZBW574VHBA3HMSQERYFBTXFVRAJQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TMPdLfCbmgi3GqDDg6AUvTCFPzbCCgwTAn9JJBrYJRnbkKtpb): protocol: expected piece hash; context canceled; EOF Aug 19 16:54:40.045862330 (263h28m20.777129961s) [noise=true , address=77.90.31.11:4579 , piece_id=E72566DABDPJRFHOB2BUUUMRM4I23RO2DLIMYPRILMAJDBX54XDA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dRcuBLuT2ybWGzdkp4vyo7e37qrZk6SBpWWHty5WKRoVHp1p): protocol: expected piece hash; context canceled; EOF Aug 19 16:54:40.045955475 (263h28m20.777223116s) [noise=true , address=79.127.201.215:4352 , piece_id=DXLVEBP2YYIUME2KXIUHOCH33FH6P4MEJD7EOP6BCNEJLJ37Y3RA, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12KNKWRTR6pZfxWFcFwsEKaqGsUMBBSTBsxvmeKsV1GDfEBvQk7): protocol: write tcp 5.161.89.252:60156->79.127.201.215:4352: use of closed network connection; write tcp 5.161.89.252:60156->79.127.201.215:4352: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:60156->79.127.201.215:4352: use of closed network connection Aug 19 16:54:40.045990340 (263h28m20.777257991s) [noise=true , address=77.90.31.19:4858 , piece_id=H2FLBEJW5FVFFY7Y7MI3KTQR36W5NAAVWZ3XJ4DQFP27DPH2DSVQ, seg_pos={0 0}, upload=273997]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1F7WDweCEEWG1kxAAQpJPcigEHsrjuqpuGhGTzNPPYTsXEX43c): protocol: expected piece hash; context canceled; EOF Aug 19 16:54:40.046041047 (263h28m20.777308707s) [seg_pos={0 0}, upload=273997]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 19 16:54:40.046213360 (263h28m20.77748101s) [upload=273997]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 19 16:54:40.046222577 (263h28m20.777490227s) [upload=273997]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 16:54:40.507200222 (263h28m21.238467872s) [upload=273997]: Done uploading segments... Aug 19 16:54:40.507219628 (263h28m21.238487279s) [upload=273997]: Upload finished. err: <nil> Duration: 670.793178ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 390.871839ms FAIL (2025-08-19 17:19:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 390.896074ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 14.467088106s FAIL (2025-08-19 17:18:49 UTC) orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 14.467117542s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 3 out of 60 is failed (threshold=3) [X............................................X...........X..] duration: 297.210337ms FAIL (2025-08-19 17:18:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 19 17:16:50.148435305 (263h50m30.879702956s) [upload=274393]: Starting upload Aug 19 17:16:50.148525875 (263h50m30.879793526s) [upload=274393]: Done starting upload Aug 19 17:16:50.148615374 (263h50m30.879883023s) [upload=274393]: Uploading segments... Aug 19 17:16:50.148667010 (263h50m30.879934661s) [upload=274393]: Got next segment. Inline: true Aug 19 17:16:50.148674014 (263h50m30.879941663s) [upload=274393]: Waiting for error group managing segments... Aug 19 17:16:50.148720332 (263h50m30.879987981s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 19 17:16:50.148725801 (263h50m30.879993452s) [upload=274393]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 19 17:16:50.148729298 (263h50m30.879996948s) [upload=274393]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 19 17:16:50.221372078 (263h50m30.952639728s) [upload=274393]: Done uploading segments... Aug 19 17:16:50.221390462 (263h50m30.952658113s) [upload=274393]: Upload finished. err: metaclient: Too Many Requests Duration:* 297.232919ms core uplink us-select-1 us1-placement-20 metainfo-projects

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)