Orbiter History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.594952679s FAIL (2025-09-22 06:09:42 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.594979299s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X.X.......XXX....X....................................] duration: 557.287405ms FAIL (2025-09-22 06:09:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 557.308092ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.565462796s FAIL (2025-09-22 06:09:27 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.565490058s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 415.541074ms FAIL (2025-09-22 06:09:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 415.566421ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......X.X.......XXX....X...................................] duration: 460.344222ms FAIL (2025-09-22 06:09:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 460.35975ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.27143397s FAIL (2025-09-22 06:08:56 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.271458075s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.09493786s FAIL (2025-09-22 06:08:54 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.094962136s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...X..XX....................................................] duration: 653.759901ms FAIL (2025-09-22 06:08:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 653.779187ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.528946489s FAIL (2025-09-22 06:08:30 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.528971296s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........X.X.......XXX....X..................................] duration: 456.805479ms FAIL (2025-09-22 06:08:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 456.826999ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 394.727498ms FAIL (2025-09-22 06:07:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 394.756782ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X...X..XX...................................................] duration: 386.161824ms FAIL (2025-09-22 06:07:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 386.183565ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.171196838s FAIL (2025-09-22 06:07:41 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.171220935s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X...X..XX..................................................] duration: 362.301576ms FAIL (2025-09-22 06:07:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 362.315211ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X.X.......XXX....X.................................] duration: 408.832736ms FAIL (2025-09-22 06:07:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 408.869946ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.142744756s FAIL (2025-09-22 06:07:21 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.142777006s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.089131746s FAIL (2025-09-22 06:07:09 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.089151163s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 405.74387ms FAIL (2025-09-22 06:07:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 405.768427ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 2.204923629s FAIL (2025-09-22 06:06:55 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.2049453s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X.X...X..XX.................................................] duration: 701.916612ms FAIL (2025-09-22 06:06:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 701.934034ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 4.950118976s FAIL (2025-09-22 06:06:49 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.950144303s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........X.X.......XXX....X................................] duration: 414.491567ms FAIL (2025-09-22 06:06:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 414.506334ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 377.109325ms FAIL (2025-09-22 06:06:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 377.134382ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 4.986835128s FAIL (2025-09-22 06:06:08 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.986860325s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 484.6319ms FAIL (2025-09-22 06:05:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 484.654463ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 2.186800531s FAIL (2025-09-22 06:05:49 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.186827842s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X.X...X..XX................................................] duration: 511.028724ms FAIL (2025-09-22 06:05:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 511.053009ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [X..........X.X.......XXX....X...............................] duration: 412.068309ms FAIL (2025-09-22 06:05:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 412.086313ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.151215676s FAIL (2025-09-22 06:05:10 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.151243037s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X.X...X..XX...............................................] duration: 639.964637ms FAIL (2025-09-22 06:04:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 639.988263ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 2.176907853s FAIL (2025-09-22 06:04:44 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.17693205s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 484.569461ms FAIL (2025-09-22 06:04:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 484.595941ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [XX..........X.X.......XXX....X..............................] duration: 395.607249ms FAIL (2025-09-22 06:04:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 395.635251ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 4.967964612s FAIL (2025-09-22 06:04:05 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 4.96799004s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XXX..........X.X.......XXX....X.............................] duration: 722.237943ms FAIL (2025-09-22 06:03:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 722.258833ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 2.150648339s FAIL (2025-09-22 06:03:50 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.150667816s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X.X...X..XX..............................................] duration: 453.462542ms FAIL (2025-09-22 06:03:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 453.487459ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 395.003154ms FAIL (2025-09-22 06:03:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 395.025296ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.124901572s FAIL (2025-09-22 06:03:30 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.124927902s core uplink ap1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 2.476839598s FAIL (2025-09-22 06:03:29 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.476858604s core uplink ap1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.XXX..........X.X.......XXX....X............................] duration: 415.424118ms FAIL (2025-09-22 06:03:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 415.581964ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X.X...X..XX.............................................] duration: 444.371455ms FAIL (2025-09-22 06:03:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 444.396982ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 2.511771672s FAIL (2025-09-22 06:02:57 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.511801258s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 380.38488ms FAIL (2025-09-22 06:02:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 380.405909ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..XXX..........X.X.......XXX....X...........................] duration: 431.456302ms FAIL (2025-09-22 06:02:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 431.494343ms edge gateway us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 3.074748774s FAIL (2025-09-22 06:02:28 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 3.074774633s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 379.708171ms FAIL (2025-09-22 06:02:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== 9934 (567h15m36.673385032s) [upload=499235]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:24:01.320603951 (567h15m36.67338905s) [upload=499235]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:24:01.344325988 (567h15m36.697111096s) [seg_pos={0 0}, upload=499235]: Begin upload segment... Sep 22 05:24:01.344389868 (567h15m36.697174966s) [seg_pos={0 0}, upload=499235]: Done begin upload segment. Sep 22 05:24:01.344399326 (567h15m36.697184434s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344412581 (567h15m36.697197679s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344474567 (567h15m36.697259665s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344712894 (567h15m36.697497992s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344738953 (567h15m36.697524061s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344964736 (567h15m36.697749845s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.344993961 (567h15m36.697779059s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345215879 (567h15m36.698000976s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345430722 (567h15m36.698215819s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345644282 (567h15m36.69842938s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.345848054 (567h15m36.698633162s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346066714 (567h15m36.698851823s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346387497 (567h15m36.699172605s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346607670 (567h15m36.699392778s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.346816642 (567h15m36.69960175s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347041865 (567h15m36.699826973s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347256778 (567h15m36.700041886s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.347697735 (567h15m36.700482833s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Uploading piece... Sep 22 05:24:01.414950461 (567h15m36.767735559s) [upload=499235]: Next returned nil segment Sep 22 05:24:01.414961502 (567h15m36.76774661s) [upload=499235]: Waiting for error group managing segments... Sep 22 05:24:01.422347973 (567h15m36.775133081s) [noise=true , address=145.79.227.33:4896 , piece_id=QMOKLQUWHGHSDFWDR7IZY6BMHHJ3TWADGSMGT6TRPVMG266LOVFA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.422665028 (567h15m36.775450126s) [noise=true , address=45.91.116.10:4093 , piece_id=BIPXKZ6MS73NJRTFOF5Y2AC2QBTLBDBJUX7CZDXRPKGTU63GNDVQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423494805 (567h15m36.776279913s) [noise=true , address=77.90.31.11:4586 , piece_id=OINTNI22SDJBJVQRV7CPZTXMQLUZF3QSIT4H4GCYFSRJ3UQX2PJA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.423635299 (567h15m36.776420407s) [noise=true , address=77.90.31.11:4561 , piece_id=A4Z3F2IYHUPXJW6XPGL2W5AZ6ML3FTZWBJ2D23C2GY2BEPLU3JBQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424085374 (567h15m36.776870482s) [noise=true , address=145.79.227.39:4029 , piece_id=SN2QGNKQSZHG74IRY2D2J7ULI5Z5HZT5EAHRDIK4FZ4ZOJPPSAMA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424273466 (567h15m36.777058574s) [noise=true , address=145.79.227.46:4624 , piece_id=Q5L7C66B6ARTZZE32WUE6YCG4SDDVOOYEZSWJJH6JQCEPUGPQU7Q, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.424401617 (567h15m36.777186715s) [noise=true , address=145.79.227.24:4586 , piece_id=HBTOQADVMWPZ2NXJ6RWIJUQKRS6V74DKMYPQABLL5JZ44JP4QPLQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.425203882 (567h15m36.777988981s) [noise=true , address=77.90.31.21:4895 , piece_id=EQI7JWUW7OANDOU2WC2CEEF45DJ2DRM6TIFDLBAXB7OMSNYT7WZQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.427714755 (567h15m36.780499863s) [noise=true , address=145.79.227.43:4168 , piece_id=6Z4UAVVRKHSKOLFGTROTRVZKSBYVJNKXUOQC4XLYKODP6RZVV7GQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.437243556 (567h15m36.790028664s) [noise=true , address=45.91.116.11:4136 , piece_id=QISYHS33KDXPH4FTPM4ASCPWDQ36TUZ2L4CFDYTHMTGOUC6AWLGA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.440363601 (567h15m36.793148709s) [noise=true , address=77.90.31.10:4538 , piece_id=JAFZFM3Y7IC4LM6XIYZO5MTRFD3IFPXU6PX6TLWELGRNYWUUE42A, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:01.939873873 (567h15m37.292658971s) [noise=true , address=66.23.193.175:4559 , piece_id=QFQ6YXNDC7ZUILECHCFMYUHJO57ODHP7ZJNMXTWBJKYAU5NEKFDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: <nil> Sep 22 05:24:17.802234801 (567h15m53.155019909s) [noise=true , address=145.79.227.11:4733 , piece_id=7JQLVSKC5ZSUDVP7ZXXX2CLJZPJK5NFMGB7QDQU7W76G5I7PCTHA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:12T7L64rXL1enKykmSocFM5ZfVeVkKJWgrHVXWbkxayuuW2FXce, address:145.79.227.11:4733): protocol: rpc: dial tcp 145.79.227.11:4733: connect: no route to host; conn closed: op Sep 22 05:25:01.306924852 (567h16m36.65970996s) [noise=true , address=145.79.227.3:4152 , piece_id=WJ66BMJSGW65ZG2FBVUD4LE5APCXFHWOITLMNOH3UPAGDFGUAVDA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:122rsZGFcSywYKfw4DPU7duA9TGtj1iXmxS4ewGGGSXGRf7Q7mX, address:145.79.227.3:4152): protocol: rpc: dial tcp 145.79.227.3:4152: i/o timeout; conn closed: op Sep 22 05:25:01.306960940 (567h16m36.659746048s) [noise=true , address=145.79.227.17:4982 , piece_id=IUXTYHGNYVXLUNOCLUWFUG4XXN7OXTRIL4GVISQOQGU2BKTNZEBA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:17EBpouVrJJDho24tucF5w43aFMqjeMviU3VoRCoL2b2Ho8WNU, address:145.79.227.17:4982): protocol: rpc: dial tcp 145.79.227.17:4982: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.306989373 (567h16m36.659774471s) [noise=true , address=145.79.227.15:4896 , piece_id=SLTDLXF4V2WLVH54BH2AQH7MY6CDQW5O6SLU22LBU3YOKMK6UBFQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1XD8SwxzT9F6zcfBtCL6dCtBm3hyPMXgo5ooJ8oM2UL1wdnT7f, address:145.79.227.15:4896): protocol: rpc: dial tcp 145.79.227.15:4896: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307006856 (567h16m36.659791964s) [noise=true , address=145.79.227.19:4107 , piece_id=XB4PUXGBEQA3CIVYCTDX3AXVKST6ECBW5URDOXBHJI7QZ3WUAUXA, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:1UeLnfo3bu6nCXkQDATe2yNhQEn6kZXicg6CFU9mGB37c1QJk4, address:145.79.227.19:4107): protocol: rpc: dial tcp 145.79.227.19:4107: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:25:01.307076026 (567h16m36.659861124s) [noise=true , address=145.79.227.13:4825 , piece_id=TZPZQQR5RZCLH4S2C3WA7OTTLD5B452QBGTC3QYIZGQLP2YMWJYQ, seg_pos={0 0}, upload=499235]: Done uploading piece. err: ecclient: upload failed (node:121Bec7R14ujGEdoUqFTikngLVLb6jmLAcHkeBAaCkhZ5dr6APb, address:145.79.227.13:4825): protocol: rpc: dial tcp 145.79.227.13:4825: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 22 05:25:01.307090433 (567h16m36.659875541s) [seg_pos={0 0}, upload=499235]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Sep 22 05:25:01.321172108 (567h16m36.673957206s) [upload=499235]: Done uploading segments... Sep 22 05:25:01.321194440 (567h16m36.673979548s) [upload=499235]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 379.734129ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.415670481s FAIL (2025-09-22 06:02:21 UTC) orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded Duration: 5.415697982s core uplink ap1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X.X...X..XX............................................] duration: 445.105154ms FAIL (2025-09-22 06:02:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 05:31:26.835749533 (567h23m2.188534662s) [upload=499337]: Starting upload Sep 22 05:31:26.835842508 (567h23m2.188627636s) [upload=499337]: Done starting upload Sep 22 05:31:26.836008360 (567h23m2.188793499s) [upload=499337]: Uploading segments... Sep 22 05:31:26.836536982 (567h23m2.18932211s) [upload=499337]: Got next segment. Inline: false Sep 22 05:31:26.836604177 (567h23m2.189389316s) [upload=499337]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 05:31:26.836610950 (567h23m2.189396078s) [upload=499337]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 05:31:26.856276449 (567h23m2.209061577s) [seg_pos={0 0}, upload=499337]: Begin upload segment... Sep 22 05:31:26.856314771 (567h23m2.209099899s) [seg_pos={0 0}, upload=499337]: Done begin upload segment. Sep 22 05:31:26.856372709 (567h23m2.209157838s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856423275 (567h23m2.209208413s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856888909 (567h23m2.209674037s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.856930226 (567h23m2.209715354s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857336809 (567h23m2.210121948s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857426228 (567h23m2.210211355s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857842308 (567h23m2.210627436s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.857850002 (567h23m2.210635131s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858275311 (567h23m2.211060449s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858418950 (567h23m2.211204089s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858708293 (567h23m2.211493421s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.858912537 (567h23m2.211697665s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859330581 (567h23m2.212115709s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.859801715 (567h23m2.212586844s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.860342481 (567h23m2.213127609s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:26.861314445 (567h23m2.214099573s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Uploading piece... Sep 22 05:31:27.009344801 (567h23m2.362129929s) [upload=499337]: Next returned nil segment Sep 22 05:31:27.009357354 (567h23m2.362142472s) [upload=499337]: Waiting for error group managing segments... Sep 22 05:31:27.018549517 (567h23m2.371334634s) [noise=true , address=45.91.116.10:4083 , piece_id=QEXMDWEXTNAK6G2D57HXRTL7IDRBW4YTJU5CRNKEBZCU2KB4KQYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.019540115 (567h23m2.372325234s) [noise=true , address=45.91.116.12:4226 , piece_id=LQBHGIFJU5G6X236IZDZ5HAAE72JQOHGJNMHBVLQMBOFPWJRA6YQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020276638 (567h23m2.373061756s) [noise=true , address=77.90.31.15:4697 , piece_id=4UZLG7IG7LNI6IIEOQNFCPVDYUL5EBPHUUADBEL7HTIG65UUOGBA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.020786695 (567h23m2.373571803s) [noise=true , address=77.90.31.12:4592 , piece_id=QPHSGW6YISMAVOKK42UVIA34Y5XVG7VRWOH4YNS4J47MB6BRYQOA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021133024 (567h23m2.373918133s) [noise=true , address=45.91.116.12:4227 , piece_id=H72VXS2S24APAIVE235B43FQ2QQOKZBALS7M5YOYGRZ46CBNODOQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021696293 (567h23m2.37448141s) [noise=true , address=77.90.31.21:4878 , piece_id=RFAS3WXHSBDUZB4N65FPRCLQMTLLUERT5MIKDJESWZNAKSAJ5FQQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.021726819 (567h23m2.374511927s) [noise=true , address=45.91.116.5:4297 , piece_id=LLTELCMD7KAR5BWPFCW3GN5QT2EV7H55Y7H63ZPLCM22LEUPCAXA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022237469 (567h23m2.375022586s) [noise=true , address=77.90.31.10:4520 , piece_id=6J2K4Z4HSVFUO5NLXECOJNGVZYKZFCKE75MUTFEVEGGT6OMHNRAA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022647758 (567h23m2.375432876s) [noise=true , address=45.91.116.6:4975 , piece_id=G66KGEC4PS55COOC6DDB5WJK5BDE2KOLLAE5TEJPXOTYK7NDSLKA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.022916483 (567h23m2.3757016s) [noise=true , address=77.90.31.16:4734 , piece_id=XX7JTCK2IIK4WHGKTJRH6L3BKIAXX26Z3BOUP5Z27WZGY3GPO74Q, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023451627 (567h23m2.376236745s) [noise=true , address=77.90.31.15:4698 , piece_id=7464MCTJYS4IYXQBIAGMPBL6UWXHF776SEAYKGLCDJOP55B5LOPA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:27.023639529 (567h23m2.376424648s) [noise=true , address=77.90.31.19:4841 , piece_id=II4IFUVQJ53X2LBYSMU5J67IT55XC2F5WWBDVGUHJUA3ZRC4WAYA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: <nil> Sep 22 05:31:30.479420128 (567h23m5.832205236s) [noise=true , address=145.79.227.16:4912 , piece_id=ZK7F3H22XCV2SRCYR4CEGAOZU2JVYJLXXGKLNYUAVKDBZ72PDYUA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: rpc: dial tcp 145.79.227.16:4912: connect: no route to host; manager closed: conn closed: op Sep 22 05:31:56.821587941 (567h23m32.174373059s) [noise=true , address=145.79.227.11:4731 , piece_id=TIH6T4DZR3KFORJGSR4XUG236CFJPW4WUFKKCN76WWWFNTUHFIHA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821645730 (567h23m32.174430847s) [noise=true , address=145.79.227.11:4732 , piece_id=CLCC77ZBAZTFJUPLFTNRLQPA3HWJLKC26K5BUARPFXFVIF7GXYIA, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: rpc: dial tcp 145.79.227.11:4732: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 05:31:56.821723946 (567h23m32.174509064s) [noise=true , address=145.79.227.5:4222 , piece_id=DSO23N5SNXL44PITFQK5PJCGHQ5R2IZISJ3MAOCZNYYAA5OGNHTQ, seg_pos={0 0}, upload=499337]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: rpc: dial tcp 145.79.227.5:4222: i/o timeout; context deadline exceeded Sep 22 05:31:56.821747761 (567h23m32.174532879s) [seg_pos={0 0}, upload=499337]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 05:31:56.838365960 (567h23m32.191151059s) [upload=499337]: Done uploading segments... Sep 22 05:31:56.838377933 (567h23m32.191163031s) [upload=499337]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 445.120673ms core uplink us-select-1
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 4.558240807s FAIL (2025-09-22 06:02:10 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 4.558262288s core uplink ap1

Orbiter Check History

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)