Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...........X...X.....X....XXX..........X.X.......XXX....X...] duration: 402.932572ms FAIL (2025-09-22 05:44:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 402.95311ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X.X..X..................X.........................] duration: 409.167704ms FAIL (2025-09-22 05:44:26 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 409.205615ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.X.X.......XXX.X.....X.......X.X...X..XX....................] duration: 376.299385ms FAIL (2025-09-22 05:44:23 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 376.325924ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [............X...X.....X....XXX..........X.X.......XXX....X..] duration: 398.720877ms FAIL (2025-09-22 05:43:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 398.816757ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..X.X.......XXX.X.....X.......X.X...X..XX...................] duration: 430.809101ms FAIL (2025-09-22 05:43:55 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 430.831172ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X.X..X..................X........................] duration: 342.040919ms FAIL (2025-09-22 05:43:21 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 342.114898ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [...X.X.......XXX.X.....X.......X.X...X..XX..................] duration: 427.874102ms FAIL (2025-09-22 05:42:52 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 427.894451ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.............X...X.....X....XXX..........X.X.......XXX....X.] duration: 433.521451ms FAIL (2025-09-22 05:42:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 433.563101ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [....X.X.......XXX.X.....X.......X.X...X..XX.................] duration: 579.728316ms FAIL (2025-09-22 05:42:34 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 579.744515ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..............X...X.....X....XXX..........X.X.......XXX....X] duration: 30.001173364s FAIL (2025-09-22 05:42:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001195264s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.....X.X.......XXX.X.....X.......X.X...X..XX................] duration: 470.020563ms FAIL (2025-09-22 05:42:07 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 470.039609ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X.X..X..................X.......................] duration: 386.501734ms FAIL (2025-09-22 05:42:06 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 386.524346ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [......X.X.......XXX.X.....X.......X.X...X..XX...............] duration: 776.146366ms FAIL (2025-09-22 05:41:44 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 776.168396ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X.X..X..................X......................] duration: 435.058246ms FAIL (2025-09-22 05:41:27 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 435.091889ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.......X.X.......XXX.X.....X.......X.X...X..XX..............] duration: 388.429233ms FAIL (2025-09-22 05:41:15 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 388.450682ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...............X...X.....X....XXX..........X.X.......XXX....] duration: 424.137349ms FAIL (2025-09-22 05:40:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 424.156144ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X.......X.X.......XXX.X.....X.......X.X...X..XX.............] duration: 430.700467ms FAIL (2025-09-22 05:40:35 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 430.719602ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X.X..X..................X.....................] duration: 493.618002ms FAIL (2025-09-22 05:40:16 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 493.648509ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.544209042s FAIL (2025-09-22 05:40:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.544232726s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X.......X.X.......XXX.X.....X.......X.X...X..XX............] duration: 695.194395ms FAIL (2025-09-22 05:39:59 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 695.222116ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X.X..X..................X....................] duration: 375.428311ms FAIL (2025-09-22 05:39:46 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 375.460241ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 4.950885934s FAIL (2025-09-22 05:39:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.950905601s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [................X...X.....X....XXX..........X.X.......XXX...] duration: 422.9387ms FAIL (2025-09-22 05:39:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 422.954419ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.027192996s FAIL (2025-09-22 05:39:18 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.027213035s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X.X..X..................X...................] duration: 387.110116ms FAIL (2025-09-22 05:39:16 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 387.139271ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.................X...X.....X....XXX..........X.X.......XXX..] duration: 428.806852ms FAIL (2025-09-22 05:39:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 428.850885ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..X.......X.X.......XXX.X.....X.......X.X...X..XX...........] duration: 500.663986ms FAIL (2025-09-22 05:39:00 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 500.684473ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X.X..X..................X..................] duration: 374.404349ms FAIL (2025-09-22 05:38:46 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 374.425379ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................X.X..X..................X.................] duration: 707.895063ms FAIL (2025-09-22 05:38:24 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 707.919909ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...X.......X.X.......XXX.X.....X.......X.X...X..XX..........] duration: 458.040509ms FAIL (2025-09-22 05:38:23 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 458.069143ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.110268145s FAIL (2025-09-22 05:38:11 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.110294254s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..................X...X.....X....XXX..........X.X.......XXX.] duration: 427.826162ms FAIL (2025-09-22 05:37:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 427.84119ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [X..................X...X.....X....XXX..........X.X.......XXX] duration: 30.001103029s FAIL (2025-09-22 05:37:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001120753s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................X.X..X..................X................] duration: 653.718853ms FAIL (2025-09-22 05:37:29 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 653.739551ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X...X.......X.X.......XXX.X.....X.......X.X...X..XX.........] duration: 387.376576ms FAIL (2025-09-22 05:37:28 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 387.397155ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 4.987349349s FAIL (2025-09-22 05:37:28 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.987368164s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X..................X...X.....X....XXX..........X.X.......XX] duration: 30.001033076s FAIL (2025-09-22 05:37:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001045098s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.X...X.......X.X.......XXX.X.....X.......X.X...X..XX........] duration: 455.439488ms FAIL (2025-09-22 05:37:12 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 455.482979ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.032675608s FAIL (2025-09-22 05:36:58 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.032694273s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..X...X.......X.X.......XXX.X.....X.......X.X...X..XX.......] duration: 406.402033ms FAIL (2025-09-22 05:36:35 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 406.42697ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X..................X...X.....X....XXX..........X.X.......X] duration: 30.000798026s FAIL (2025-09-22 05:36:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000822422s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................X.X..X..................X...............] duration: 754.982777ms FAIL (2025-09-22 05:36:14 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 755.004347ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.132480485s FAIL (2025-09-22 05:35:56 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.132500492s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [...X...X.......X.X.......XXX.X.....X.......X.X...X..XX......] duration: 453.963778ms FAIL (2025-09-22 05:35:46 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 453.985849ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [....X...X.......X.X.......XXX.X.....X.......X.X...X..XX.....] duration: 459.560892ms FAIL (2025-09-22 05:35:08 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 459.585819ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.46087496s FAIL (2025-09-22 05:35:06 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.460902201s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................X.X..X..................X..............] duration: 382.943567ms FAIL (2025-09-22 05:35:00 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 382.965798ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...X..................X...X.....X....XXX..........X.X.......] duration: 454.874056ms FAIL (2025-09-22 05:34:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 454.899073ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.....X...X.......X.X.......XXX.X.....X.......X.X...X..XX....] duration: 781.899135ms FAIL (2025-09-22 05:34:24 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 781.922409ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.164773713s FAIL (2025-09-22 05:34:11 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.16479383s core uplink ap1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)