Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 364.616909ms FAIL (2025-09-22 06:02:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 364.637856ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...XXX..........X.X.......XXX....X..........................] duration: 425.197216ms FAIL (2025-09-22 06:02:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 425.214878ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 446.458923ms FAIL (2025-09-22 06:01:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 446.486044ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.531037336s FAIL (2025-09-22 06:01:26 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.531069737s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.783546566s FAIL (2025-09-22 06:01:20 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.783567344s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X.X...X..XX...........................................] duration: 429.345988ms FAIL (2025-09-22 06:01:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 429.381044ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [....XXX..........X.X.......XXX....X.........................] duration: 441.222058ms FAIL (2025-09-22 06:00:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 441.241554ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X.X...X..XX..........................................] duration: 523.949755ms FAIL (2025-09-22 06:00:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 523.976104ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 895.668242ms FAIL (2025-09-22 06:00:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 895.694161ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.281314919s FAIL (2025-09-22 06:00:38 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.281339715s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 4.953801766s FAIL (2025-09-22 06:00:32 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.953829898s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [X....XXX..........X.X.......XXX....X........................] duration: 487.571117ms FAIL (2025-09-22 06:00:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 487.591054ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.......X.X...X..XX.........................................] duration: 489.247802ms FAIL (2025-09-22 06:00:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 489.273099ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 418.66682ms FAIL (2025-09-22 06:00:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 418.691917ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.22687246s FAIL (2025-09-22 05:59:56 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.226888861s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X.......X.X...X..XX........................................] duration: 661.878101ms FAIL (2025-09-22 05:59:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 661.903069ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.045874983s FAIL (2025-09-22 05:59:37 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.045906621s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.229805635s FAIL (2025-09-22 05:59:34 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.229822878s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.X....XXX..........X.X.......XXX....X.......................] duration: 428.529948ms FAIL (2025-09-22 05:59:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 428.553041ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 4.982770137s FAIL (2025-09-22 05:59:11 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.982791356s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 407.736059ms FAIL (2025-09-22 05:59:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 407.752049ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X.......X.X...X..XX.......................................] duration: 422.702474ms FAIL (2025-09-22 05:58:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 422.755884ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.130058257s FAIL (2025-09-22 05:58:48 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.130083125s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 4.997990421s FAIL (2025-09-22 05:58:45 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.998015437s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X.......X.X...X..XX......................................] duration: 496.927023ms FAIL (2025-09-22 05:58:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 496.945448ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..X....XXX..........X.X.......XXX....X......................] duration: 418.651752ms FAIL (2025-09-22 05:58:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 418.914535ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...X....XXX..........X.X.......XXX....X.....................] duration: 723.442793ms FAIL (2025-09-22 05:57:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 723.459995ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 363.813894ms FAIL (2025-09-22 05:57:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 363.83313ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.822322963s FAIL (2025-09-22 05:57:52 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.822387846s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.32530502s FAIL (2025-09-22 05:57:43 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.325329957s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....X....XXX..........X.X.......XXX....X....................] duration: 422.488113ms FAIL (2025-09-22 05:57:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 422.508541ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X.......X.X...X..XX.....................................] duration: 597.944183ms FAIL (2025-09-22 05:57:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 597.96448ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X.......X.X...X..XX....................................] duration: 440.92371ms FAIL (2025-09-22 05:57:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 440.94505ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.210000617s FAIL (2025-09-22 05:57:06 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.210022349s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 366.212084ms FAIL (2025-09-22 05:57:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 366.233303ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 4.953390073s FAIL (2025-09-22 05:57:00 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.953418718s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 681.244815ms FAIL (2025-09-22 05:56:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 681.261997ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.19059945s FAIL (2025-09-22 05:56:43 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.190626201s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.429125699s FAIL (2025-09-22 05:56:38 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.429145145s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.....X....XXX..........X.X.......XXX....X...................] duration: 757.994072ms FAIL (2025-09-22 05:56:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 758.018727ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [X.....X.......X.X...X..XX...................................] duration: 393.613555ms FAIL (2025-09-22 05:56:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 393.642139ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 473.128834ms FAIL (2025-09-22 05:56:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 473.167927ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.238082219s FAIL (2025-09-22 05:56:00 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.238106856s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X.....X....XXX..........X.X.......XXX....X..................] duration: 437.196173ms FAIL (2025-09-22 05:55:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 437.369598ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.24548073s FAIL (2025-09-22 05:55:38 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.245509494s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.13085683s FAIL (2025-09-22 05:55:28 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.130898758s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X.....X.......X.X...X..XX..................................] duration: 372.700258ms FAIL (2025-09-22 05:55:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 372.71686ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 450.875501ms FAIL (2025-09-22 05:55:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 450.900547ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.X.....X....XXX..........X.X.......XXX....X.................] duration: 544.032331ms FAIL (2025-09-22 05:55:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 544.054171ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 4.901221577s FAIL (2025-09-22 05:55:12 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.901249789s core uplink ap1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)