Orbiter History

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X...X..X......X..................XXX.XXXXX.........X........] duration: 416.018561ms FAIL (2025-09-22 04:52:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 416.038868ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X...X..X......X..................XXX.XXXXX.........X.......] duration: 398.723445ms FAIL (2025-09-22 04:52:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 398.733514ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X.X...X..X......X..................XXX.XXXXX.........X......] duration: 470.768444ms FAIL (2025-09-22 04:52:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 470.787898ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXX...................................X.....................] duration: 539.885327ms FAIL (2025-09-22 04:51:45 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 540.02006ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.X.X...X..X......X..................XXX.XXXXX.........X.....] duration: 564.637486ms FAIL (2025-09-22 04:51:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 564.658194ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [..X.X...X..X......X..................XXX.XXXXX.........X....] duration: 422.364258ms FAIL (2025-09-22 04:50:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 422.378646ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXX...................................X....................] duration: 699.010957ms FAIL (2025-09-22 04:50:39 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 699.036776ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [...X.X...X..X......X..................XXX.XXXXX.........X...] duration: 414.068103ms FAIL (2025-09-22 04:50:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 414.08796ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXX...................................X...................] duration: 659.074471ms FAIL (2025-09-22 04:50:04 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 659.102826ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [....X.X...X..X......X..................XXX.XXXXX.........X..] duration: 422.768298ms FAIL (2025-09-22 04:49:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 422.789477ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [X..XXX...................................X..................] duration: 456.86318ms FAIL (2025-09-22 04:49:10 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 456.876955ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.....X.X...X..X......X..................XXX.XXXXX.........X.] duration: 451.065329ms FAIL (2025-09-22 04:49:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 451.079226ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.X..XXX...................................X.................] duration: 597.495538ms FAIL (2025-09-22 04:48:53 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 597.520684ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..X..XXX...................................X................] duration: 694.866864ms FAIL (2025-09-22 04:48:19 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 694.885569ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...X..XXX...................................X...............] duration: 524.103042ms FAIL (2025-09-22 04:48:00 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 524.126335ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.......X.X...X..X......X..................XXX.XXXXX.........] duration: 437.054989ms FAIL (2025-09-22 04:47:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 437.076089ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....X..XXX...................................X..............] duration: 442.578423ms FAIL (2025-09-22 04:46:47 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 442.604711ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [........X.X...X..X......X..................XXX.XXXXX........] duration: 625.857931ms FAIL (2025-09-22 04:46:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 625.999586ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.........X.X...X..X......X..................XXX.XXXXX.......] duration: 469.244712ms FAIL (2025-09-22 04:46:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 469.25878ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....X..XXX...................................X.............] duration: 468.437418ms FAIL (2025-09-22 04:46:00 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 468.458407ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..........X.X...X..X......X..................XXX.XXXXX......] duration: 442.652805ms FAIL (2025-09-22 04:45:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 442.861195ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X..XXX...................................X............] duration: 405.440131ms FAIL (2025-09-22 04:45:16 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 405.463014ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X..XXX...................................X...........] duration: 664.180718ms FAIL (2025-09-22 04:44:38 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 664.202499ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...........X.X...X..X......X..................XXX.XXXXX.....] duration: 429.194938ms FAIL (2025-09-22 04:44:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 429.216629ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [........X..XXX...................................X..........] duration: 689.740172ms FAIL (2025-09-22 04:43:49 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 689.766953ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [............X.X...X..X......X..................XXX.XXXXX....] duration: 459.609253ms FAIL (2025-09-22 04:43:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.625042ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X........X..XXX...................................X.........] duration: 385.804573ms FAIL (2025-09-22 04:43:11 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 385.830883ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X........X..XXX...................................X........] duration: 488.29801ms FAIL (2025-09-22 04:42:30 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 488.323217ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.............X.X...X..X......X..................XXX.XXXXX...] duration: 460.580366ms FAIL (2025-09-22 04:42:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 460.60444ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..X........X..XXX...................................X.......] duration: 507.065249ms FAIL (2025-09-22 04:41:17 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 507.084145ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..............X.X...X..X......X..................XXX.XXXXX..] duration: 888.120453ms FAIL (2025-09-22 04:40:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 888.135342ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...X........X..XXX...................................X......] duration: 395.849884ms FAIL (2025-09-22 04:40:47 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 395.875022ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [...............X.X...X..X......X..................XXX.XXXXX.] duration: 425.030122ms FAIL (2025-09-22 04:40:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 425.047494ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [................X.X...X..X......X..................XXX.XXXXX] duration: 30.000421378s FAIL (2025-09-22 04:40:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000447808s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....X........X..XXX...................................X.....] duration: 419.186245ms FAIL (2025-09-22 04:39:47 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 419.222934ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.................X.X...X..X......X..................XXX.XXXX] duration: 30.002190203s FAIL (2025-09-22 04:39:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.002211775s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..................X.X...X..X......X..................XXX.XXX] duration: 30.000763271s FAIL (2025-09-22 04:39:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000795672s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....X........X..XXX...................................X....] duration: 761.653054ms FAIL (2025-09-22 04:38:44 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 761.674786ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [X..................X.X...X..X......X..................XXX.XX] duration: 30.001697515s FAIL (2025-09-22 04:38:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001720187s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [XX..................X.X...X..X......X..................XXX.X] duration: 30.00111022s FAIL (2025-09-22 04:37:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001136041s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......X........X..XXX...................................X...] duration: 716.03635ms FAIL (2025-09-22 04:37:35 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 716.070675ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......X........X..XXX...................................X..] duration: 459.26062ms FAIL (2025-09-22 04:37:16 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 459.285957ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.XX..................X.X...X..X......X..................XXX.] duration: 491.57438ms FAIL (2025-09-22 04:36:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 491.594928ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........X........X..XXX...................................X.] duration: 424.494908ms FAIL (2025-09-22 04:36:46 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 424.51673ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........X........X..XXX...................................X] duration: 30.015673292s FAIL (2025-09-22 04:36:42 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 22 04:36:12.644832629 (566h27m47.997617726s) [upload=498538]: Starting upload Sep 22 04:36:12.644902188 (566h27m47.997687296s) [upload=498538]: Done starting upload Sep 22 04:36:12.644968543 (566h27m47.997753651s) [upload=498538]: Uploading segments... Sep 22 04:36:12.645120949 (566h27m47.997906047s) [upload=498538]: Got next segment. Inline: false Sep 22 04:36:12.645149963 (566h27m47.997935071s) [upload=498538]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:36:12.645153330 (566h27m47.997938438s) [upload=498538]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:36:12.671594820 (566h27m48.024379958s) [seg_pos={0 0}, upload=498538]: Begin upload segment... Sep 22 04:36:12.671637109 (566h27m48.024422217s) [seg_pos={0 0}, upload=498538]: Done begin upload segment. Sep 22 04:36:12.671683396 (566h27m48.024468494s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.671789284 (566h27m48.024574392s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672167214 (566h27m48.024952352s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672317737 (566h27m48.025102835s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672532309 (566h27m48.025317417s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.672939864 (566h27m48.025724962s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673170226 (566h27m48.025955334s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673501818 (566h27m48.026286926s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673873606 (566h27m48.026658714s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.673883694 (566h27m48.026668802s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674173469 (566h27m48.026958577s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674192495 (566h27m48.026977593s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674502076 (566h27m48.027287184s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674523706 (566h27m48.027308814s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.674899711 (566h27m48.027684819s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.675288862 (566h27m48.02807396s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Uploading piece... Sep 22 04:36:12.754015098 (566h27m48.106800206s) [upload=498538]: Next returned nil segment Sep 22 04:36:12.754026128 (566h27m48.106811236s) [upload=498538]: Waiting for error group managing segments... Sep 22 04:36:12.762538753 (566h27m48.115323851s) [noise=true , address=45.91.116.4:4939 , piece_id=B2KT7EFM7BMVLHPQD35UWN352VYWNQQ7C66IZTXGN2VNFCQPWQXA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763691737 (566h27m48.116476826s) [noise=true , address=45.91.116.10:4083 , piece_id=7UHK5QBWHPR47YNPXPWB4P2NFNSZH2RXACWSXW26TNB2CZR2LAMA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.763851938 (566h27m48.116637036s) [noise=true , address=77.90.31.17:4772 , piece_id=I6EOUCOVQABAQM2DCS7QGXOGRYAAYWMLWUDRXDP4EUME32GVDUJA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764119320 (566h27m48.116904407s) [noise=true , address=45.91.116.8:4047 , piece_id=HIN5PY4OCEVZULCCZRA2IB52BSHM7SOZ6IMMAB6OGIZ4VMWKUQ2Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764344512 (566h27m48.11712962s) [noise=true , address=45.91.116.6:4974 , piece_id=VCGDW3XLL2EE3FEMJZDLCEJ54GLG73EQBPWZ62TKJ54SYY5MMDBQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764450341 (566h27m48.117235428s) [noise=true , address=45.91.116.9:4261 , piece_id=O2HJ5NUG2I7IRF6MVMFH4VWY47R3L7GF4EA7ZCYLATHQGMI7DYDA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764690291 (566h27m48.117475399s) [noise=true , address=77.90.31.16:4735 , piece_id=LOZIHPZLJBLPG4VLFU443EH3M2IC5TE7RZ5BM7LHQL4GVXROI2BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.764770231 (566h27m48.117555319s) [noise=true , address=77.90.31.16:4734 , piece_id=ID2PAF5VIXHG2FX4XN3GFECPM43HCQ5VUQPFDNKHNR34UXGIIT5Q, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765165502 (566h27m48.117950601s) [noise=true , address=77.90.31.14:4663 , piece_id=KXOH7EYNYSGFOT4WWO4LZ3EJPJSN3TV326EJ2N3O3GR4QQDQ3ZMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765188967 (566h27m48.117974075s) [noise=true , address=77.90.31.18:4806 , piece_id=23A3S3W6IXULKCW6EILAMAII353EH4V7BIBPRAPZGGE4DHVKNYMQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765508888 (566h27m48.118293985s) [noise=true , address=77.90.31.15:4698 , piece_id=ZBIDGJBBGYB63KYMS6HK4H534T6VJJIYTONCTD3TXLCLHURBP3BQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:12.765532983 (566h27m48.118318081s) [noise=true , address=77.90.31.19:4841 , piece_id=RZKK73XV5E4VBZ73QIPV7UN2Q3HLWS55GOIVCTHCGNQS2FQEWLUA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: <nil> Sep 22 04:36:42.632309187 (566h28m17.985094306s) [noise=true , address=145.79.227.3:4151 , piece_id=HEEJZ5WRA3XODAVULTWCYJ3MLI7DAD2DTOFYVKVSJ7AEEAIJPPPQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632392333 (566h28m17.985177462s) [noise=true , address=145.79.227.15:4873 , piece_id=WCQ6GQRQDOXNLG7SMZVLKQ2RUQRR4GYSROVIUUYVQ3UGSP3ZS5QA, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: rpc: dial tcp 145.79.227.15:4873: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632448048 (566h28m17.985233176s) [noise=true , address=145.79.227.7:4294 , piece_id=GH2PNKFYARMA67TLPFQZS5CMCLP6ONK3TIH4T3UFKDJGTTUKXNDQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, address:145.79.227.7:4294): protocol: rpc: dial tcp 145.79.227.7:4294: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632483284 (566h28m17.985268402s) [noise=true , address=145.79.227.16:4910 , piece_id=IGZZ3OIBLBS4T3GLGDBOMJYLYEREU3OISWPZO22BXMLLQI2VN7DQ, seg_pos={0 0}, upload=498538]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: rpc: dial tcp 145.79.227.16:4910: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:36:42.632529260 (566h28m17.985314378s) [seg_pos={0 0}, upload=498538]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 22 04:36:42.646011962 (566h28m17.99879709s) [upload=498538]: Done uploading segments... Sep 22 04:36:42.646038512 (566h28m17.99882365s) [upload=498538]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.015694331s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..XX..................X.X...X..X......X..................XXX] duration: 30.001112456s FAIL (2025-09-22 04:36:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00112565s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...XX..................X.X...X..X......X..................XX] duration: 30.001026544s FAIL (2025-09-22 04:35:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001219546s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........X........X..XXX...................................] duration: 478.590594ms FAIL (2025-09-22 04:35:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 478.60981ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [....XX..................X.X...X..X......X..................X] duration: 30.001594352s FAIL (2025-09-22 04:35:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001613258s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [...........X........X..XXX..................................] duration: 829.9036ms FAIL (2025-09-22 04:34:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 22 04:10:24.100440694 (566h1m59.453225822s) [upload=498156]: Starting upload Sep 22 04:10:24.100520314 (566h1m59.453305442s) [upload=498156]: Done starting upload Sep 22 04:10:24.100735217 (566h1m59.453520345s) [upload=498156]: Uploading segments... Sep 22 04:10:24.101026633 (566h1m59.453811761s) [upload=498156]: Got next segment. Inline: false Sep 22 04:10:24.101075957 (566h1m59.453861095s) [upload=498156]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 22 04:10:24.101081457 (566h1m59.453866585s) [upload=498156]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 22 04:10:24.141321941 (566h1m59.49410704s) [seg_pos={0 0}, upload=498156]: Begin upload segment... Sep 22 04:10:24.141357568 (566h1m59.494142666s) [seg_pos={0 0}, upload=498156]: Done begin upload segment. Sep 22 04:10:24.141385170 (566h1m59.494170278s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141445864 (566h1m59.494231002s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141678511 (566h1m59.494463608s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141927137 (566h1m59.494712245s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.141990595 (566h1m59.494775724s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142153943 (566h1m59.494939051s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142381640 (566h1m59.495166768s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142409462 (566h1m59.49519456s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142691211 (566h1m59.495476308s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142793723 (566h1m59.495578851s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.142933555 (566h1m59.495718663s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143158097 (566h1m59.495943205s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143165702 (566h1m59.495950829s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143383330 (566h1m59.496168438s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143536176 (566h1m59.496321315s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.143973899 (566h1m59.496759026s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Uploading piece... Sep 22 04:10:24.239968480 (566h1m59.592753588s) [upload=498156]: Next returned nil segment Sep 22 04:10:24.239980803 (566h1m59.592765911s) [upload=498156]: Waiting for error group managing segments... Sep 22 04:10:24.248891123 (566h1m59.601676222s) [noise=true , address=45.91.116.6:4975 , piece_id=76USR62GWBSXUU2MS342YKEKS4J3OMRCRMFZMKHJ6MPC3IL23NDA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249484668 (566h1m59.602269806s) [noise=true , address=77.90.31.16:4733 , piece_id=HAZAR4TOACTT3ASTIHB5246ENO5HYLS2GNFHNV7XH65JGGMIYUQQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.249704590 (566h1m59.602489698s) [noise=true , address=45.91.116.14:4189 , piece_id=I37DGFASGWXDN4MJB5A62HJET7EUHZ732ZXFEI6RDRTLUU4M4MOQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250108327 (566h1m59.602893456s) [noise=true , address=45.91.116.14:4190 , piece_id=KC567SOI5VQXAHLS6Y4SHGU5VXEK2U4MSUPTUNW6I2OLHKWOJWRQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250133304 (566h1m59.602918402s) [noise=true , address=77.90.31.14:4662 , piece_id=CICQLONUXLNZHKXQDF4K57YFRG3OYLVYJCA2RSGMRMPE2N7OEEAQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250596404 (566h1m59.603381502s) [noise=true , address=45.91.116.9:4262 , piece_id=O2ZZYVZF2DZFNQZ37S2RIBNMQEXWYINCT4QVIMNRWRWMOQRWZWLQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250743099 (566h1m59.603528227s) [noise=true , address=77.90.31.21:4879 , piece_id=LSMQGD6R76PGVJZ6IZCHQCKU3NSPG7LFTRHPXBNUPDTCES2GGD6Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.250985423 (566h1m59.603770521s) [noise=true , address=77.90.31.21:4877 , piece_id=FB2TADVIS6CEVVAAPTCZCUCHIKRJUASN53J6BBTORBEMJYHJTGYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.251295306 (566h1m59.604080433s) [noise=true , address=45.91.116.13:4155 , piece_id=X64HHNW74DGDPDYB5MDT57D5SMMDVSXD6JGGJ6BAK4OPHCQMZZMA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:24.252309960 (566h1m59.605095057s) [noise=true , address=77.90.31.15:4697 , piece_id=KARHFNGBGATUZD5UROSKUSPJDHLBAVJYPZLETQVZYH62B5RZQGWA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: <nil> Sep 22 04:10:25.114124764 (566h2m0.466909893s) [noise=true , address=145.79.227.5:4223 , piece_id=3QABJHOGWIA4YA6SEWNGWTCF3SMPMYSIS2JWRQY2FNZ5QA7OSL7Q, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: rpc: dial tcp 145.79.227.5:4223: connect: no route to host; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op Sep 22 04:10:25.338224247 (566h2m0.691009384s) [noise=true , address=145.79.227.18:4005 , piece_id=TPPQ4QW4UO5DIVPZFKWHHUI23YFVFOVTOQ7EVYDVJ7NTOYDPKPCQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: rpc: dial tcp 145.79.227.18:4005: connect: no route to host; conn closed: op Sep 22 04:10:26.389719283 (566h2m1.742504391s) [noise=true , address=145.79.227.11:4730 , piece_id=JIFGUBU5GSE2ADQVP77HV4QDQKAPZN7ER2Y2ULD4PPYCIRXQ7UYQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: rpc: dial tcp 145.79.227.11:4730: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088552400 (566h2m29.441337538s) [noise=true , address=145.79.227.7:4293 , piece_id=QGQBZIXR5U6XGPWFZOBKD24P4GRYDOYSMCCBCOVHG7DERFINQ6AQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 22 04:10:54.088698834 (566h2m29.441483963s) [noise=true , address=145.79.227.6:4259 , piece_id=5LHCOTXW3VW2UAEX2WL5TNFF6FN6NMY5PLSAS3ITWDPLHPRDOXGA, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088742276 (566h2m29.441527404s) [noise=true , address=145.79.227.15:4873 , piece_id=S3KVWWHCCEOECHSU3L3NZ54FOATSPC55WC6V3RXJDK73YGEDQOHQ, seg_pos={0 0}, upload=498156]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: context deadline exceeded; context deadline exceeded Sep 22 04:10:54.088842734 (566h2m29.441627862s) [seg_pos={0 0}, upload=498156]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 22 04:10:54.101418053 (566h2m29.45420318s) [upload=498156]: Done uploading segments... Sep 22 04:10:54.101435134 (566h2m29.454220273s) [upload=498156]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 829.921445ms core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)