Orbiter History

Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 635191 reports (Page 11171 of 12704)

checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [......................................XX.XXXX.XXX..X..XX....] duration: 401.320967ms FAIL (2025-09-19 20:25:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 401.338471ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.......................................XX.XXXX.XXX..X..XX...] duration: 436.387349ms FAIL (2025-09-19 20:24:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 436.406845ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [......................................X..X.XXXXXX.X..XX..X..] duration: 781.019377ms FAIL (2025-09-19 20:24:24 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== .227.16:4911 , piece_id=TXDTUB2CGSAVRV3ZXTKJDKA2PSB3OE72HHKMYBKQ3DCWHZXPFEFA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.845531674 (510h14m34.198316802s) [noise=true , address=77.90.31.10:4518 , piece_id=PBRMRQUEUAAMHMY6NBAIY7UV6UUZQ5XKDMKT2T6B4JZXRKUQFKAQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846032524 (510h14m34.198817663s) [noise=true , address=45.91.116.12:4225 , piece_id=WZYJUA4QTIZL54ELEAX3QRDBSXJNYYAK6OXFLY6KZSRLUUPG3NAQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846043646 (510h14m34.198828773s) [noise=true , address=45.91.116.8:4048 , piece_id=GK3IVYIYDZ7CUEBOVTIWRY7USYNQT33CVVTKGIIZD4PMC452UOMA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846429770 (510h14m34.199214897s) [noise=true , address=145.79.227.7:4295 , piece_id=E2S3QUZJAHYIWX2A4SJWKBBAQCLT32Q2OGQH23E5FGPKKZINP2TA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846507636 (510h14m34.199292763s) [noise=true , address=145.79.227.13:4804 , piece_id=PWOMBVAGSNTH6CYJAXFQIX3YIQTLIYTAV6FRB3IF3ZW2HUKB4D6A, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846821395 (510h14m34.199606533s) [noise=true , address=45.91.116.9:4262 , piece_id=QSPIL7E4LILHAW2VPVATPPGWD577ZDLCY334DQCVOMR5YQJZXFGA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846942151 (510h14m34.19972729s) [noise=true , address=77.90.31.11:4555 , piece_id=A2OUQISOOJQLFJBFX4HNTKALW5JYKXGKSKRP2XL3Y3T7NPO2K6RQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.847326151 (510h14m34.20011128s) [noise=true , address=45.91.116.4:4938 , piece_id=FNZZM4NUB2P2VXH7U7EIY25742SQJZKENW6PNN6JL65TYQI566IQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.847771057 (510h14m34.200556185s) [noise=true , address=77.90.31.14:4663 , piece_id=WIUFXMNP7CUQJXDSVY5GZ65DJV22QJZ3TDECU73PG3YHWTIPWSMA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.848334084 (510h14m34.201119212s) [noise=true , address=45.91.116.5:4298 , piece_id=KEQ6RCQBQVDNQ63GUULW3GRT7FKXW52NWJLFA3AB2SDQU4HZYCXQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.849219025 (510h14m34.202004153s) [noise=true , address=77.90.31.15:4697 , piece_id=N5DR2AUKQ7AXTGIHD5KOBRMLF3AZBU4N2GUDOJTTESTA5OVFFNAQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.849977238 (510h14m34.202762366s) [noise=true , address=45.91.116.14:4190 , piece_id=GWES6XL46PPSZYB7AUAFLMYGIYXPNEWRAZAJKIGV6BGIXTRI4VEQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.851690302 (510h14m34.204475441s) [noise=true , address=77.90.31.16:4734 , piece_id=RQA6CLX2HNCSHM5J7MOFAOGKUHZDMINODHULHGV6RR4MEU5KFBMA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.851789309 (510h14m34.204574446s) [noise=true , address=145.79.227.11:4732 , piece_id=SBFP3NXNERYUD3QUSOIBVFTCNWX7O3DNBW7CFDDGMAGPQB3IT4ZA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.852388703 (510h14m34.205173841s) [noise=true , address=145.79.227.3:4151 , piece_id=AKKMKNHU5ODNNESGJOVGCD4JKVYIOW2ROTFCG2AZ3S4JWDXVYOAA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.945885961 (510h14m34.29867109s) [upload=449152]: Next returned nil segment Sep 19 20:22:58.945901340 (510h14m34.298686468s) [upload=449152]: Waiting for error group managing segments... Sep 19 20:22:58.961933689 (510h14m34.314718827s) [noise=true , address=145.79.227.3:4151 , piece_id=AKKMKNHU5ODNNESGJOVGCD4JKVYIOW2ROTFCG2AZ3S4JWDXVYOAA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:58.977887148 (510h14m34.330672277s) [noise=true , address=145.79.227.13:4804 , piece_id=PWOMBVAGSNTH6CYJAXFQIX3YIQTLIYTAV6FRB3IF3ZW2HUKB4D6A, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:58.978498557 (510h14m34.331283684s) [noise=true , address=145.79.227.7:4295 , piece_id=E2S3QUZJAHYIWX2A4SJWKBBAQCLT32Q2OGQH23E5FGPKKZINP2TA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:58.979160138 (510h14m34.331945266s) [noise=true , address=145.79.227.16:4911 , piece_id=TXDTUB2CGSAVRV3ZXTKJDKA2PSB3OE72HHKMYBKQ3DCWHZXPFEFA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:59.206612232 (510h14m34.559397361s) [noise=true , address=145.79.227.11:4732 , piece_id=SBFP3NXNERYUD3QUSOIBVFTCNWX7O3DNBW7CFDDGMAGPQB3IT4ZA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:23:28.729557262 (510h15m4.08234236s) [noise=true , address=45.91.116.12:4225 , piece_id=WZYJUA4QTIZL54ELEAX3QRDBSXJNYYAK6OXFLY6KZSRLUUPG3NAQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729641309 (510h15m4.082426417s) [noise=true , address=45.91.116.4:4938 , piece_id=FNZZM4NUB2P2VXH7U7EIY25742SQJZKENW6PNN6JL65TYQI566IQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729738503 (510h15m4.082523611s) [noise=true , address=45.91.116.8:4048 , piece_id=GK3IVYIYDZ7CUEBOVTIWRY7USYNQT33CVVTKGIIZD4PMC452UOMA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 5.161.89.252:20230->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729753361 (510h15m4.082538459s) [noise=true , address=77.90.31.16:4734 , piece_id=RQA6CLX2HNCSHM5J7MOFAOGKUHZDMINODHULHGV6RR4MEU5KFBMA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729769100 (510h15m4.082554199s) [noise=true , address=77.90.31.14:4663 , piece_id=WIUFXMNP7CUQJXDSVY5GZ65DJV22QJZ3TDECU73PG3YHWTIPWSMA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 5.161.89.252:30010->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729804527 (510h15m4.082589625s) [noise=true , address=77.90.31.11:4555 , piece_id=A2OUQISOOJQLFJBFX4HNTKALW5JYKXGKSKRP2XL3Y3T7NPO2K6RQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729845785 (510h15m4.082630882s) [noise=true , address=45.91.116.9:4262 , piece_id=QSPIL7E4LILHAW2VPVATPPGWD577ZDLCY334DQCVOMR5YQJZXFGA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729885860 (510h15m4.082670957s) [noise=true , address=45.91.116.14:4190 , piece_id=GWES6XL46PPSZYB7AUAFLMYGIYXPNEWRAZAJKIGV6BGIXTRI4VEQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:29536->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729894906 (510h15m4.082680014s) [noise=true , address=45.91.116.5:4298 , piece_id=KEQ6RCQBQVDNQ63GUULW3GRT7FKXW52NWJLFA3AB2SDQU4HZYCXQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 5.161.89.252:25794->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729918451 (510h15m4.082703549s) [noise=true , address=77.90.31.15:4697 , piece_id=N5DR2AUKQ7AXTGIHD5KOBRMLF3AZBU4N2GUDOJTTESTA5OVFFNAQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 5.161.89.252:54484->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729924962 (510h15m4.08271007s) [noise=true , address=77.90.31.10:4518 , piece_id=PBRMRQUEUAAMHMY6NBAIY7UV6UUZQ5XKDMKT2T6B4JZXRKUQFKAQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 5.161.89.252:15830->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729958696 (510h15m4.082743804s) [seg_pos={0 0}, upload=449152]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Sep 19 20:23:28.746501923 (510h15m4.099287021s) [upload=449152]: Done uploading segments... Sep 19 20:23:28.746512543 (510h15m4.099297641s) [upload=449152]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 781.044544ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [........................................XX.XXXX.XXX..X..XX..] duration: 547.679221ms FAIL (2025-09-19 20:23:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 547.707544ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.......................................X..X.XXXXXX.X..XX..X.] duration: 7.622608594s FAIL (2025-09-19 20:23:36 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== .227.16:4911 , piece_id=TXDTUB2CGSAVRV3ZXTKJDKA2PSB3OE72HHKMYBKQ3DCWHZXPFEFA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.845531674 (510h14m34.198316802s) [noise=true , address=77.90.31.10:4518 , piece_id=PBRMRQUEUAAMHMY6NBAIY7UV6UUZQ5XKDMKT2T6B4JZXRKUQFKAQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846032524 (510h14m34.198817663s) [noise=true , address=45.91.116.12:4225 , piece_id=WZYJUA4QTIZL54ELEAX3QRDBSXJNYYAK6OXFLY6KZSRLUUPG3NAQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846043646 (510h14m34.198828773s) [noise=true , address=45.91.116.8:4048 , piece_id=GK3IVYIYDZ7CUEBOVTIWRY7USYNQT33CVVTKGIIZD4PMC452UOMA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846429770 (510h14m34.199214897s) [noise=true , address=145.79.227.7:4295 , piece_id=E2S3QUZJAHYIWX2A4SJWKBBAQCLT32Q2OGQH23E5FGPKKZINP2TA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846507636 (510h14m34.199292763s) [noise=true , address=145.79.227.13:4804 , piece_id=PWOMBVAGSNTH6CYJAXFQIX3YIQTLIYTAV6FRB3IF3ZW2HUKB4D6A, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846821395 (510h14m34.199606533s) [noise=true , address=45.91.116.9:4262 , piece_id=QSPIL7E4LILHAW2VPVATPPGWD577ZDLCY334DQCVOMR5YQJZXFGA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846942151 (510h14m34.19972729s) [noise=true , address=77.90.31.11:4555 , piece_id=A2OUQISOOJQLFJBFX4HNTKALW5JYKXGKSKRP2XL3Y3T7NPO2K6RQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.847326151 (510h14m34.20011128s) [noise=true , address=45.91.116.4:4938 , piece_id=FNZZM4NUB2P2VXH7U7EIY25742SQJZKENW6PNN6JL65TYQI566IQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.847771057 (510h14m34.200556185s) [noise=true , address=77.90.31.14:4663 , piece_id=WIUFXMNP7CUQJXDSVY5GZ65DJV22QJZ3TDECU73PG3YHWTIPWSMA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.848334084 (510h14m34.201119212s) [noise=true , address=45.91.116.5:4298 , piece_id=KEQ6RCQBQVDNQ63GUULW3GRT7FKXW52NWJLFA3AB2SDQU4HZYCXQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.849219025 (510h14m34.202004153s) [noise=true , address=77.90.31.15:4697 , piece_id=N5DR2AUKQ7AXTGIHD5KOBRMLF3AZBU4N2GUDOJTTESTA5OVFFNAQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.849977238 (510h14m34.202762366s) [noise=true , address=45.91.116.14:4190 , piece_id=GWES6XL46PPSZYB7AUAFLMYGIYXPNEWRAZAJKIGV6BGIXTRI4VEQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.851690302 (510h14m34.204475441s) [noise=true , address=77.90.31.16:4734 , piece_id=RQA6CLX2HNCSHM5J7MOFAOGKUHZDMINODHULHGV6RR4MEU5KFBMA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.851789309 (510h14m34.204574446s) [noise=true , address=145.79.227.11:4732 , piece_id=SBFP3NXNERYUD3QUSOIBVFTCNWX7O3DNBW7CFDDGMAGPQB3IT4ZA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.852388703 (510h14m34.205173841s) [noise=true , address=145.79.227.3:4151 , piece_id=AKKMKNHU5ODNNESGJOVGCD4JKVYIOW2ROTFCG2AZ3S4JWDXVYOAA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.945885961 (510h14m34.29867109s) [upload=449152]: Next returned nil segment Sep 19 20:22:58.945901340 (510h14m34.298686468s) [upload=449152]: Waiting for error group managing segments... Sep 19 20:22:58.961933689 (510h14m34.314718827s) [noise=true , address=145.79.227.3:4151 , piece_id=AKKMKNHU5ODNNESGJOVGCD4JKVYIOW2ROTFCG2AZ3S4JWDXVYOAA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:58.977887148 (510h14m34.330672277s) [noise=true , address=145.79.227.13:4804 , piece_id=PWOMBVAGSNTH6CYJAXFQIX3YIQTLIYTAV6FRB3IF3ZW2HUKB4D6A, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:58.978498557 (510h14m34.331283684s) [noise=true , address=145.79.227.7:4295 , piece_id=E2S3QUZJAHYIWX2A4SJWKBBAQCLT32Q2OGQH23E5FGPKKZINP2TA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:58.979160138 (510h14m34.331945266s) [noise=true , address=145.79.227.16:4911 , piece_id=TXDTUB2CGSAVRV3ZXTKJDKA2PSB3OE72HHKMYBKQ3DCWHZXPFEFA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:59.206612232 (510h14m34.559397361s) [noise=true , address=145.79.227.11:4732 , piece_id=SBFP3NXNERYUD3QUSOIBVFTCNWX7O3DNBW7CFDDGMAGPQB3IT4ZA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:23:28.729557262 (510h15m4.08234236s) [noise=true , address=45.91.116.12:4225 , piece_id=WZYJUA4QTIZL54ELEAX3QRDBSXJNYYAK6OXFLY6KZSRLUUPG3NAQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729641309 (510h15m4.082426417s) [noise=true , address=45.91.116.4:4938 , piece_id=FNZZM4NUB2P2VXH7U7EIY25742SQJZKENW6PNN6JL65TYQI566IQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729738503 (510h15m4.082523611s) [noise=true , address=45.91.116.8:4048 , piece_id=GK3IVYIYDZ7CUEBOVTIWRY7USYNQT33CVVTKGIIZD4PMC452UOMA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 5.161.89.252:20230->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729753361 (510h15m4.082538459s) [noise=true , address=77.90.31.16:4734 , piece_id=RQA6CLX2HNCSHM5J7MOFAOGKUHZDMINODHULHGV6RR4MEU5KFBMA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729769100 (510h15m4.082554199s) [noise=true , address=77.90.31.14:4663 , piece_id=WIUFXMNP7CUQJXDSVY5GZ65DJV22QJZ3TDECU73PG3YHWTIPWSMA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 5.161.89.252:30010->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729804527 (510h15m4.082589625s) [noise=true , address=77.90.31.11:4555 , piece_id=A2OUQISOOJQLFJBFX4HNTKALW5JYKXGKSKRP2XL3Y3T7NPO2K6RQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729845785 (510h15m4.082630882s) [noise=true , address=45.91.116.9:4262 , piece_id=QSPIL7E4LILHAW2VPVATPPGWD577ZDLCY334DQCVOMR5YQJZXFGA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729885860 (510h15m4.082670957s) [noise=true , address=45.91.116.14:4190 , piece_id=GWES6XL46PPSZYB7AUAFLMYGIYXPNEWRAZAJKIGV6BGIXTRI4VEQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:29536->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729894906 (510h15m4.082680014s) [noise=true , address=45.91.116.5:4298 , piece_id=KEQ6RCQBQVDNQ63GUULW3GRT7FKXW52NWJLFA3AB2SDQU4HZYCXQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 5.161.89.252:25794->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729918451 (510h15m4.082703549s) [noise=true , address=77.90.31.15:4697 , piece_id=N5DR2AUKQ7AXTGIHD5KOBRMLF3AZBU4N2GUDOJTTESTA5OVFFNAQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 5.161.89.252:54484->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729924962 (510h15m4.08271007s) [noise=true , address=77.90.31.10:4518 , piece_id=PBRMRQUEUAAMHMY6NBAIY7UV6UUZQ5XKDMKT2T6B4JZXRKUQFKAQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 5.161.89.252:15830->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729958696 (510h15m4.082743804s) [seg_pos={0 0}, upload=449152]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Sep 19 20:23:28.746501923 (510h15m4.099287021s) [upload=449152]: Done uploading segments... Sep 19 20:23:28.746512543 (510h15m4.099297641s) [upload=449152]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.622621928s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [.........................................XX.XXXX.XXX..X..XX.] duration: 1.012976567s FAIL (2025-09-19 20:23:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1.012996253s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..........................................XX.XXXX.XXX..X..XX] duration: 30.000388915s FAIL (2025-09-19 20:23:33 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000470108s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [........................................X..X.XXXXXX.X..XX..X] duration: 30.018657094s FAIL (2025-09-19 20:23:28 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== .227.16:4911 , piece_id=TXDTUB2CGSAVRV3ZXTKJDKA2PSB3OE72HHKMYBKQ3DCWHZXPFEFA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.845531674 (510h14m34.198316802s) [noise=true , address=77.90.31.10:4518 , piece_id=PBRMRQUEUAAMHMY6NBAIY7UV6UUZQ5XKDMKT2T6B4JZXRKUQFKAQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846032524 (510h14m34.198817663s) [noise=true , address=45.91.116.12:4225 , piece_id=WZYJUA4QTIZL54ELEAX3QRDBSXJNYYAK6OXFLY6KZSRLUUPG3NAQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846043646 (510h14m34.198828773s) [noise=true , address=45.91.116.8:4048 , piece_id=GK3IVYIYDZ7CUEBOVTIWRY7USYNQT33CVVTKGIIZD4PMC452UOMA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846429770 (510h14m34.199214897s) [noise=true , address=145.79.227.7:4295 , piece_id=E2S3QUZJAHYIWX2A4SJWKBBAQCLT32Q2OGQH23E5FGPKKZINP2TA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846507636 (510h14m34.199292763s) [noise=true , address=145.79.227.13:4804 , piece_id=PWOMBVAGSNTH6CYJAXFQIX3YIQTLIYTAV6FRB3IF3ZW2HUKB4D6A, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846821395 (510h14m34.199606533s) [noise=true , address=45.91.116.9:4262 , piece_id=QSPIL7E4LILHAW2VPVATPPGWD577ZDLCY334DQCVOMR5YQJZXFGA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.846942151 (510h14m34.19972729s) [noise=true , address=77.90.31.11:4555 , piece_id=A2OUQISOOJQLFJBFX4HNTKALW5JYKXGKSKRP2XL3Y3T7NPO2K6RQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.847326151 (510h14m34.20011128s) [noise=true , address=45.91.116.4:4938 , piece_id=FNZZM4NUB2P2VXH7U7EIY25742SQJZKENW6PNN6JL65TYQI566IQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.847771057 (510h14m34.200556185s) [noise=true , address=77.90.31.14:4663 , piece_id=WIUFXMNP7CUQJXDSVY5GZ65DJV22QJZ3TDECU73PG3YHWTIPWSMA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.848334084 (510h14m34.201119212s) [noise=true , address=45.91.116.5:4298 , piece_id=KEQ6RCQBQVDNQ63GUULW3GRT7FKXW52NWJLFA3AB2SDQU4HZYCXQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.849219025 (510h14m34.202004153s) [noise=true , address=77.90.31.15:4697 , piece_id=N5DR2AUKQ7AXTGIHD5KOBRMLF3AZBU4N2GUDOJTTESTA5OVFFNAQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.849977238 (510h14m34.202762366s) [noise=true , address=45.91.116.14:4190 , piece_id=GWES6XL46PPSZYB7AUAFLMYGIYXPNEWRAZAJKIGV6BGIXTRI4VEQ, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.851690302 (510h14m34.204475441s) [noise=true , address=77.90.31.16:4734 , piece_id=RQA6CLX2HNCSHM5J7MOFAOGKUHZDMINODHULHGV6RR4MEU5KFBMA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.851789309 (510h14m34.204574446s) [noise=true , address=145.79.227.11:4732 , piece_id=SBFP3NXNERYUD3QUSOIBVFTCNWX7O3DNBW7CFDDGMAGPQB3IT4ZA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.852388703 (510h14m34.205173841s) [noise=true , address=145.79.227.3:4151 , piece_id=AKKMKNHU5ODNNESGJOVGCD4JKVYIOW2ROTFCG2AZ3S4JWDXVYOAA, seg_pos={0 0}, upload=449152]: Uploading piece... Sep 19 20:22:58.945885961 (510h14m34.29867109s) [upload=449152]: Next returned nil segment Sep 19 20:22:58.945901340 (510h14m34.298686468s) [upload=449152]: Waiting for error group managing segments... Sep 19 20:22:58.961933689 (510h14m34.314718827s) [noise=true , address=145.79.227.3:4151 , piece_id=AKKMKNHU5ODNNESGJOVGCD4JKVYIOW2ROTFCG2AZ3S4JWDXVYOAA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:58.977887148 (510h14m34.330672277s) [noise=true , address=145.79.227.13:4804 , piece_id=PWOMBVAGSNTH6CYJAXFQIX3YIQTLIYTAV6FRB3IF3ZW2HUKB4D6A, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:58.978498557 (510h14m34.331283684s) [noise=true , address=145.79.227.7:4295 , piece_id=E2S3QUZJAHYIWX2A4SJWKBBAQCLT32Q2OGQH23E5FGPKKZINP2TA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:58.979160138 (510h14m34.331945266s) [noise=true , address=145.79.227.16:4911 , piece_id=TXDTUB2CGSAVRV3ZXTKJDKA2PSB3OE72HHKMYBKQ3DCWHZXPFEFA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:22:59.206612232 (510h14m34.559397361s) [noise=true , address=145.79.227.11:4732 , piece_id=SBFP3NXNERYUD3QUSOIBVFTCNWX7O3DNBW7CFDDGMAGPQB3IT4ZA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: <nil> Sep 19 20:23:28.729557262 (510h15m4.08234236s) [noise=true , address=45.91.116.12:4225 , piece_id=WZYJUA4QTIZL54ELEAX3QRDBSXJNYYAK6OXFLY6KZSRLUUPG3NAQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729641309 (510h15m4.082426417s) [noise=true , address=45.91.116.4:4938 , piece_id=FNZZM4NUB2P2VXH7U7EIY25742SQJZKENW6PNN6JL65TYQI566IQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: rpc: dial tcp 45.91.116.4:4938: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729738503 (510h15m4.082523611s) [noise=true , address=45.91.116.8:4048 , piece_id=GK3IVYIYDZ7CUEBOVTIWRY7USYNQT33CVVTKGIIZD4PMC452UOMA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: read tcp 5.161.89.252:20230->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729753361 (510h15m4.082538459s) [noise=true , address=77.90.31.16:4734 , piece_id=RQA6CLX2HNCSHM5J7MOFAOGKUHZDMINODHULHGV6RR4MEU5KFBMA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729769100 (510h15m4.082554199s) [noise=true , address=77.90.31.14:4663 , piece_id=WIUFXMNP7CUQJXDSVY5GZ65DJV22QJZ3TDECU73PG3YHWTIPWSMA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 5.161.89.252:30010->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729804527 (510h15m4.082589625s) [noise=true , address=77.90.31.11:4555 , piece_id=A2OUQISOOJQLFJBFX4HNTKALW5JYKXGKSKRP2XL3Y3T7NPO2K6RQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729845785 (510h15m4.082630882s) [noise=true , address=45.91.116.9:4262 , piece_id=QSPIL7E4LILHAW2VPVATPPGWD577ZDLCY334DQCVOMR5YQJZXFGA, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: rpc: dial tcp 45.91.116.9:4262: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729885860 (510h15m4.082670957s) [noise=true , address=45.91.116.14:4190 , piece_id=GWES6XL46PPSZYB7AUAFLMYGIYXPNEWRAZAJKIGV6BGIXTRI4VEQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG, address:45.91.116.14:4190): protocol: read tcp 5.161.89.252:29536->45.91.116.14:4190: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729894906 (510h15m4.082680014s) [noise=true , address=45.91.116.5:4298 , piece_id=KEQ6RCQBQVDNQ63GUULW3GRT7FKXW52NWJLFA3AB2SDQU4HZYCXQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 5.161.89.252:25794->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729918451 (510h15m4.082703549s) [noise=true , address=77.90.31.15:4697 , piece_id=N5DR2AUKQ7AXTGIHD5KOBRMLF3AZBU4N2GUDOJTTESTA5OVFFNAQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: read tcp 5.161.89.252:54484->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729924962 (510h15m4.08271007s) [noise=true , address=77.90.31.10:4518 , piece_id=PBRMRQUEUAAMHMY6NBAIY7UV6UUZQ5XKDMKT2T6B4JZXRKUQFKAQ, seg_pos={0 0}, upload=449152]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 5.161.89.252:15830->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:23:28.729958696 (510h15m4.082743804s) [seg_pos={0 0}, upload=449152]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Sep 19 20:23:28.746501923 (510h15m4.099287021s) [upload=449152]: Done uploading segments... Sep 19 20:23:28.746512543 (510h15m4.099297641s) [upload=449152]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.018675138s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........................................XX.XXXX.XXX..X..X] duration: 30.001573962s FAIL (2025-09-19 20:23:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00158887s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.........................................X..X.XXXXXX.X..XX..] duration: 414.698136ms FAIL (2025-09-19 20:22:22 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 16.748451808s) [seg_pos={0 0}, upload=449116]: Begin upload segment... Sep 19 20:20:41.395703289 (510h12m16.748488397s) [seg_pos={0 0}, upload=449116]: Done begin upload segment. Sep 19 20:20:41.395724368 (510h12m16.748509476s) [noise=true , address=77.90.31.13:4628 , piece_id=SUNQ57OZ6R2UUID7TGIM3KUQNVD5P7GIDPD3PBWPLVQQTWC5DTIA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.395771045 (510h12m16.748556174s) [noise=true , address=145.79.227.18:4005 , piece_id=TJBL265ZHLM3K3OYCWN2VS6WXJUD23CTDWCIV3YB3USSUSJM7FUA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396054347 (510h12m16.748839444s) [noise=true , address=45.91.116.5:4298 , piece_id=DSJFWJOAWZHLCAQLKWKVT4MJSXPLHPPX4XNJ47QIEMQUUJBGNOKA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396274740 (510h12m16.749059868s) [noise=true , address=145.79.227.6:4257 , piece_id=2HSIK4QBMK4BI7QVSMTMB66PBLLQVL3BIXGLNKO4QL3JCAYHS24Q, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396287524 (510h12m16.749072632s) [noise=true , address=145.79.227.3:4150 , piece_id=QZASEOJ2IY2ZZY24VPTMFEVHKKEKZICS4ULBAKTX3MPPMB5J4TIQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396534699 (510h12m16.749319797s) [noise=true , address=145.79.227.14:4838 , piece_id=YLT6MXE5FTJFLXWF3YLP2ID6CO6IJOYV3FCNT3DGJXH6N5B6FNZQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396697103 (510h12m16.749482231s) [noise=true , address=145.79.227.5:4223 , piece_id=CEKOPXMW6UE7JW6THNVF24YWU2NXIYGDZK5Z6SDPFNCIM3EYJVIA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396811307 (510h12m16.749596405s) [noise=true , address=145.79.227.17:4970 , piece_id=ROR3Z2MX5KII3C732AGTVTOEGSHHY2NUW5TLWRW5AWQPQKLV3NMQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397084860 (510h12m16.749869968s) [noise=true , address=45.91.116.6:4974 , piece_id=CLAHGKNUQIUJL7EYY4JRBV3OAMTG7WWT76LM4FYVNBHUAZJTR4MA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397129003 (510h12m16.749914131s) [noise=true , address=45.91.116.14:4191 , piece_id=SK3S6XO2UXMR5CKR4IG2YG64FEUW7ZWTMZVL6N7IY6D5GDGF5SNQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397308120 (510h12m16.750093228s) [noise=true , address=45.91.116.13:4153 , piece_id=AOFHJL4NYRRJTL4IH6RFECIVQ3VMWEEMJ2PABLJZ5OEPJFN4Z2OA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397496884 (510h12m16.750282012s) [noise=true , address=77.90.31.21:4879 , piece_id=4XZOVX46GEWYDDKUUQFHBDVYNPYTEMWKQE4KY53LKCJDLLTFFK3Q, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397521640 (510h12m16.750306739s) [noise=true , address=45.91.116.13:4155 , piece_id=4IAVRPPZMDGMXVGRVUS6OZR5CN74NA47RIPJ22P5A3EAX72FBFYQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397878780 (510h12m16.750663908s) [noise=true , address=77.90.31.16:4734 , piece_id=32SZORESHDUGXAOSXMSS3ATA4ZGZXZ3JEFXGYDB6F23VFNSAZ5BQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.398245199 (510h12m16.751030337s) [noise=true , address=45.91.116.7:4011 , piece_id=A5OT6NADSY3QI3F3TSBXNEFY67LHKSB66IY24OQA6FE3GURUWIBQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.399043166 (510h12m16.751828294s) [noise=true , address=145.79.227.18:4007 , piece_id=4XPBZAR3VLXPFKT3XSCD6RH2TP2ZUPHZGU5Q4HAATAMLDRPKRYDQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.475227474 (510h12m16.828012582s) [upload=449116]: Next returned nil segment Sep 19 20:20:41.475239687 (510h12m16.828024795s) [upload=449116]: Waiting for error group managing segments... Sep 19 20:20:41.482692043 (510h12m16.835477151s) [noise=true , address=145.79.227.14:4838 , piece_id=YLT6MXE5FTJFLXWF3YLP2ID6CO6IJOYV3FCNT3DGJXH6N5B6FNZQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.483554812 (510h12m16.836339941s) [noise=true , address=145.79.227.3:4150 , piece_id=QZASEOJ2IY2ZZY24VPTMFEVHKKEKZICS4ULBAKTX3MPPMB5J4TIQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.483711787 (510h12m16.836496885s) [noise=true , address=145.79.227.5:4223 , piece_id=CEKOPXMW6UE7JW6THNVF24YWU2NXIYGDZK5Z6SDPFNCIM3EYJVIA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.484125042 (510h12m16.83691017s) [noise=true , address=145.79.227.6:4257 , piece_id=2HSIK4QBMK4BI7QVSMTMB66PBLLQVL3BIXGLNKO4QL3JCAYHS24Q, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.484408956 (510h12m16.837194064s) [noise=true , address=145.79.227.17:4970 , piece_id=ROR3Z2MX5KII3C732AGTVTOEGSHHY2NUW5TLWRW5AWQPQKLV3NMQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.487799267 (510h12m16.840584395s) [noise=true , address=145.79.227.18:4007 , piece_id=4XPBZAR3VLXPFKT3XSCD6RH2TP2ZUPHZGU5Q4HAATAMLDRPKRYDQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.496468526 (510h12m16.849253634s) [noise=true , address=145.79.227.18:4005 , piece_id=TJBL265ZHLM3K3OYCWN2VS6WXJUD23CTDWCIV3YB3USSUSJM7FUA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:21:11.350285156 (510h12m46.703070264s) [noise=true , address=45.91.116.6:4974 , piece_id=CLAHGKNUQIUJL7EYY4JRBV3OAMTG7WWT76LM4FYVNBHUAZJTR4MA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op Sep 19 20:21:11.350601260 (510h12m46.703386368s) [noise=true , address=45.91.116.13:4153 , piece_id=AOFHJL4NYRRJTL4IH6RFECIVQ3VMWEEMJ2PABLJZ5OEPJFN4Z2OA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 5.161.89.252:15454->45.91.116.13:4153: use of closed network connection; write tcp 5.161.89.252:15454->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15454->45.91.116.13:4153: use of closed network connection Sep 19 20:21:11.350628050 (510h12m46.703413148s) [noise=true , address=45.91.116.7:4011 , piece_id=A5OT6NADSY3QI3F3TSBXNEFY67LHKSB66IY24OQA6FE3GURUWIBQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 5.161.89.252:44102->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350655782 (510h12m46.70344088s) [noise=true , address=77.90.31.16:4734 , piece_id=32SZORESHDUGXAOSXMSS3ATA4ZGZXZ3JEFXGYDB6F23VFNSAZ5BQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:53004->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350684005 (510h12m46.703469103s) [noise=true , address=45.91.116.14:4191 , piece_id=SK3S6XO2UXMR5CKR4IG2YG64FEUW7ZWTMZVL6N7IY6D5GDGF5SNQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12648->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350726014 (510h12m46.703511122s) [noise=true , address=77.90.31.13:4628 , piece_id=SUNQ57OZ6R2UUID7TGIM3KUQNVD5P7GIDPD3PBWPLVQQTWC5DTIA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:41372->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350731574 (510h12m46.703516673s) [noise=true , address=77.90.31.21:4879 , piece_id=4XZOVX46GEWYDDKUUQFHBDVYNPYTEMWKQE4KY53LKCJDLLTFFK3Q, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:39082->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350777671 (510h12m46.703562769s) [noise=true , address=45.91.116.13:4155 , piece_id=4IAVRPPZMDGMXVGRVUS6OZR5CN74NA47RIPJ22P5A3EAX72FBFYQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 5.161.89.252:31768->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350851069 (510h12m46.703636166s) [noise=true , address=45.91.116.5:4298 , piece_id=DSJFWJOAWZHLCAQLKWKVT4MJSXPLHPPX4XNJ47QIEMQUUJBGNOKA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 5.161.89.252:22540->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:21:11.350866257 (510h12m46.703651365s) [seg_pos={0 0}, upload=449116]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:21:11.384202757 (510h12m46.736987885s) [upload=449116]: Done uploading segments... Sep 19 20:21:11.384225620 (510h12m46.737010717s) [upload=449116]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 414.727742ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [............................................XX.XXXX.XXX..X..] duration: 444.468147ms FAIL (2025-09-19 20:21:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 444.491481ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [..........................................X..X.XXXXXX.X..XX.] duration: 494.082499ms FAIL (2025-09-19 20:21:11 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 16.748451808s) [seg_pos={0 0}, upload=449116]: Begin upload segment... Sep 19 20:20:41.395703289 (510h12m16.748488397s) [seg_pos={0 0}, upload=449116]: Done begin upload segment. Sep 19 20:20:41.395724368 (510h12m16.748509476s) [noise=true , address=77.90.31.13:4628 , piece_id=SUNQ57OZ6R2UUID7TGIM3KUQNVD5P7GIDPD3PBWPLVQQTWC5DTIA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.395771045 (510h12m16.748556174s) [noise=true , address=145.79.227.18:4005 , piece_id=TJBL265ZHLM3K3OYCWN2VS6WXJUD23CTDWCIV3YB3USSUSJM7FUA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396054347 (510h12m16.748839444s) [noise=true , address=45.91.116.5:4298 , piece_id=DSJFWJOAWZHLCAQLKWKVT4MJSXPLHPPX4XNJ47QIEMQUUJBGNOKA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396274740 (510h12m16.749059868s) [noise=true , address=145.79.227.6:4257 , piece_id=2HSIK4QBMK4BI7QVSMTMB66PBLLQVL3BIXGLNKO4QL3JCAYHS24Q, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396287524 (510h12m16.749072632s) [noise=true , address=145.79.227.3:4150 , piece_id=QZASEOJ2IY2ZZY24VPTMFEVHKKEKZICS4ULBAKTX3MPPMB5J4TIQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396534699 (510h12m16.749319797s) [noise=true , address=145.79.227.14:4838 , piece_id=YLT6MXE5FTJFLXWF3YLP2ID6CO6IJOYV3FCNT3DGJXH6N5B6FNZQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396697103 (510h12m16.749482231s) [noise=true , address=145.79.227.5:4223 , piece_id=CEKOPXMW6UE7JW6THNVF24YWU2NXIYGDZK5Z6SDPFNCIM3EYJVIA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.396811307 (510h12m16.749596405s) [noise=true , address=145.79.227.17:4970 , piece_id=ROR3Z2MX5KII3C732AGTVTOEGSHHY2NUW5TLWRW5AWQPQKLV3NMQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397084860 (510h12m16.749869968s) [noise=true , address=45.91.116.6:4974 , piece_id=CLAHGKNUQIUJL7EYY4JRBV3OAMTG7WWT76LM4FYVNBHUAZJTR4MA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397129003 (510h12m16.749914131s) [noise=true , address=45.91.116.14:4191 , piece_id=SK3S6XO2UXMR5CKR4IG2YG64FEUW7ZWTMZVL6N7IY6D5GDGF5SNQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397308120 (510h12m16.750093228s) [noise=true , address=45.91.116.13:4153 , piece_id=AOFHJL4NYRRJTL4IH6RFECIVQ3VMWEEMJ2PABLJZ5OEPJFN4Z2OA, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397496884 (510h12m16.750282012s) [noise=true , address=77.90.31.21:4879 , piece_id=4XZOVX46GEWYDDKUUQFHBDVYNPYTEMWKQE4KY53LKCJDLLTFFK3Q, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397521640 (510h12m16.750306739s) [noise=true , address=45.91.116.13:4155 , piece_id=4IAVRPPZMDGMXVGRVUS6OZR5CN74NA47RIPJ22P5A3EAX72FBFYQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.397878780 (510h12m16.750663908s) [noise=true , address=77.90.31.16:4734 , piece_id=32SZORESHDUGXAOSXMSS3ATA4ZGZXZ3JEFXGYDB6F23VFNSAZ5BQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.398245199 (510h12m16.751030337s) [noise=true , address=45.91.116.7:4011 , piece_id=A5OT6NADSY3QI3F3TSBXNEFY67LHKSB66IY24OQA6FE3GURUWIBQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.399043166 (510h12m16.751828294s) [noise=true , address=145.79.227.18:4007 , piece_id=4XPBZAR3VLXPFKT3XSCD6RH2TP2ZUPHZGU5Q4HAATAMLDRPKRYDQ, seg_pos={0 0}, upload=449116]: Uploading piece... Sep 19 20:20:41.475227474 (510h12m16.828012582s) [upload=449116]: Next returned nil segment Sep 19 20:20:41.475239687 (510h12m16.828024795s) [upload=449116]: Waiting for error group managing segments... Sep 19 20:20:41.482692043 (510h12m16.835477151s) [noise=true , address=145.79.227.14:4838 , piece_id=YLT6MXE5FTJFLXWF3YLP2ID6CO6IJOYV3FCNT3DGJXH6N5B6FNZQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.483554812 (510h12m16.836339941s) [noise=true , address=145.79.227.3:4150 , piece_id=QZASEOJ2IY2ZZY24VPTMFEVHKKEKZICS4ULBAKTX3MPPMB5J4TIQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.483711787 (510h12m16.836496885s) [noise=true , address=145.79.227.5:4223 , piece_id=CEKOPXMW6UE7JW6THNVF24YWU2NXIYGDZK5Z6SDPFNCIM3EYJVIA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.484125042 (510h12m16.83691017s) [noise=true , address=145.79.227.6:4257 , piece_id=2HSIK4QBMK4BI7QVSMTMB66PBLLQVL3BIXGLNKO4QL3JCAYHS24Q, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.484408956 (510h12m16.837194064s) [noise=true , address=145.79.227.17:4970 , piece_id=ROR3Z2MX5KII3C732AGTVTOEGSHHY2NUW5TLWRW5AWQPQKLV3NMQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.487799267 (510h12m16.840584395s) [noise=true , address=145.79.227.18:4007 , piece_id=4XPBZAR3VLXPFKT3XSCD6RH2TP2ZUPHZGU5Q4HAATAMLDRPKRYDQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:20:41.496468526 (510h12m16.849253634s) [noise=true , address=145.79.227.18:4005 , piece_id=TJBL265ZHLM3K3OYCWN2VS6WXJUD23CTDWCIV3YB3USSUSJM7FUA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: <nil> Sep 19 20:21:11.350285156 (510h12m46.703070264s) [noise=true , address=45.91.116.6:4974 , piece_id=CLAHGKNUQIUJL7EYY4JRBV3OAMTG7WWT76LM4FYVNBHUAZJTR4MA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op Sep 19 20:21:11.350601260 (510h12m46.703386368s) [noise=true , address=45.91.116.13:4153 , piece_id=AOFHJL4NYRRJTL4IH6RFECIVQ3VMWEEMJ2PABLJZ5OEPJFN4Z2OA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: write tcp 5.161.89.252:15454->45.91.116.13:4153: use of closed network connection; write tcp 5.161.89.252:15454->45.91.116.13:4153: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15454->45.91.116.13:4153: use of closed network connection Sep 19 20:21:11.350628050 (510h12m46.703413148s) [noise=true , address=45.91.116.7:4011 , piece_id=A5OT6NADSY3QI3F3TSBXNEFY67LHKSB66IY24OQA6FE3GURUWIBQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 5.161.89.252:44102->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350655782 (510h12m46.70344088s) [noise=true , address=77.90.31.16:4734 , piece_id=32SZORESHDUGXAOSXMSS3ATA4ZGZXZ3JEFXGYDB6F23VFNSAZ5BQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:53004->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350684005 (510h12m46.703469103s) [noise=true , address=45.91.116.14:4191 , piece_id=SK3S6XO2UXMR5CKR4IG2YG64FEUW7ZWTMZVL6N7IY6D5GDGF5SNQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12648->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350726014 (510h12m46.703511122s) [noise=true , address=77.90.31.13:4628 , piece_id=SUNQ57OZ6R2UUID7TGIM3KUQNVD5P7GIDPD3PBWPLVQQTWC5DTIA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:41372->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350731574 (510h12m46.703516673s) [noise=true , address=77.90.31.21:4879 , piece_id=4XZOVX46GEWYDDKUUQFHBDVYNPYTEMWKQE4KY53LKCJDLLTFFK3Q, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:39082->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350777671 (510h12m46.703562769s) [noise=true , address=45.91.116.13:4155 , piece_id=4IAVRPPZMDGMXVGRVUS6OZR5CN74NA47RIPJ22P5A3EAX72FBFYQ, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: read tcp 5.161.89.252:31768->45.91.116.13:4155: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:21:11.350851069 (510h12m46.703636166s) [noise=true , address=45.91.116.5:4298 , piece_id=DSJFWJOAWZHLCAQLKWKVT4MJSXPLHPPX4XNJ47QIEMQUUJBGNOKA, seg_pos={0 0}, upload=449116]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: read tcp 5.161.89.252:22540->45.91.116.5:4298: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Sep 19 20:21:11.350866257 (510h12m46.703651365s) [seg_pos={0 0}, upload=449116]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Sep 19 20:21:11.384202757 (510h12m46.736987885s) [upload=449116]: Done uploading segments... Sep 19 20:21:11.384225620 (510h12m46.737010717s) [upload=449116]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 494.095803ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.............................................XX.XXXX.XXX..X.] duration: 10.254612871s FAIL (2025-09-19 20:21:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 10.254630334s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [............................................X..X.XXXXXX.X..X] duration: 30.019535427s FAIL (2025-09-19 20:20:41 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 61438157 (510h11m46.614223265s) [upload=449108]: Uploading segments... Sep 19 20:20:11.261563332 (510h11m46.61434843s) [upload=449108]: Got next segment. Inline: false Sep 19 20:20:11.261616982 (510h11m46.61440208s) [upload=449108]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 20:20:11.261620940 (510h11m46.614406038s) [upload=449108]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:20:11.280920759 (510h11m46.633705847s) [seg_pos={0 0}, upload=449108]: Begin upload segment... Sep 19 20:20:11.280945926 (510h11m46.633731024s) [seg_pos={0 0}, upload=449108]: Done begin upload segment. Sep 19 20:20:11.280962567 (510h11m46.633747665s) [noise=true , address=145.79.227.18:4006 , piece_id=IU4SOF24YRAFJSEBXB4Y77WAGZIPBXSDAJHSV65Q2BETGFG3JN7Q, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.281019254 (510h11m46.633804362s) [noise=true , address=77.90.31.16:4735 , piece_id=TJYEFSUKL4BT3G3FSQAW3JUOSANB33WVVLQ54YP42V5MM3C3XLAA, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.281207546 (510h11m46.633992645s) [noise=true , address=145.79.227.4:4186 , piece_id=R6CTLQAW5GI37OXFNQDCC4TNBFFU6ZANXJSS3XYWFSMW6MJDSQFQ, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.281392213 (510h11m46.634177301s) [noise=true , address=145.79.227.6:4257 , piece_id=QYVX6WXB3HBPPG3IU3H53CLI5RV2ZO3XZJKX4GNJVLU7XSNXNWVA, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.281430175 (510h11m46.634215272s) [noise=true , address=145.79.227.15:4875 , piece_id=5NWUJI6LA67E3BMPJKYC6JOX6RFJKLOGR5CZCFOLJG3G343NI63Q, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.281568323 (510h11m46.634353422s) [noise=true , address=45.91.116.10:4083 , piece_id=RSCNBHOJHMZCYFL45UMUWMFAYHGFNHVRNOM7LKFRBTMUZKLTPANA, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.281746508 (510h11m46.634531617s) [noise=true , address=77.90.31.13:4628 , piece_id=P46LZD2I34SYMTZ233334N6HGT3TCKU2HXDHK5YBHQPR6H37S7DQ, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.281756167 (510h11m46.634541265s) [noise=true , address=145.79.227.11:4731 , piece_id=JIGUPCONJ345Y53ITWB3PPRGA6D4ARC4VUGLGFTS55DEBLJI5K5A, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.281927308 (510h11m46.634712405s) [noise=true , address=145.79.227.5:4223 , piece_id=76NGCGRCKNNO5NXQHLRWR4E6BGMJ4PRZF6GWJTI3JU3BOQCMT7RQ, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.282038366 (510h11m46.634823474s) [noise=true , address=45.91.116.5:4297 , piece_id=GMT43VQ4DSLKZB52LCEZ4M4NQLTTXABQF2HYKI7ZL2MV5RJ5MY4Q, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.282101544 (510h11m46.634886642s) [noise=true , address=145.79.227.3:4150 , piece_id=7V5JKNBJ53JUBCVH5VD525AEZKQDGYYALMGSF674RTK3NC455NHQ, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.282300137 (510h11m46.635085235s) [noise=true , address=145.79.227.19:4078 , piece_id=HWWSE7LNIE2AEDTVHHITLF4RCKP34LE7WBQ7PYV5ZNB66NSLEAAQ, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.282317509 (510h11m46.635102607s) [noise=true , address=145.79.227.14:4838 , piece_id=6HV457JG57VEG4RRQW4DFMFCIVWCRGNQRIUD5V5OWY2IEFPNU2UA, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.282478752 (510h11m46.63526385s) [noise=true , address=145.79.227.4:4185 , piece_id=2I6ZXQ6ROSRCTKFOJDIIZXLWC7CFUDE2R7KQXC5IQLUCPZP32Y6Q, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.282614778 (510h11m46.635399886s) [noise=true , address=145.79.227.19:4079 , piece_id=A7K57FKGAZM4TFBWDXTQ5OOJKKQNEX4V3Z7TIBCTD33TEGMKISDA, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.282894182 (510h11m46.63567929s) [noise=true , address=77.90.31.15:4698 , piece_id=Z6JMNN7ZFUWRQQTPPKRY4SGSU7UBXGMG6QQWLRMUE4QIM5Q4NXRQ, seg_pos={0 0}, upload=449108]: Uploading piece... Sep 19 20:20:11.350317659 (510h11m46.703102767s) [upload=449108]: Next returned nil segment Sep 19 20:20:11.350352124 (510h11m46.703137232s) [upload=449108]: Waiting for error group managing segments... Sep 19 20:20:11.358106727 (510h11m46.710891835s) [noise=true , address=145.79.227.14:4838 , piece_id=6HV457JG57VEG4RRQW4DFMFCIVWCRGNQRIUD5V5OWY2IEFPNU2UA, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.358796411 (510h11m46.711581519s) [noise=true , address=145.79.227.3:4150 , piece_id=7V5JKNBJ53JUBCVH5VD525AEZKQDGYYALMGSF674RTK3NC455NHQ, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.359338428 (510h11m46.712123536s) [noise=true , address=145.79.227.6:4257 , piece_id=QYVX6WXB3HBPPG3IU3H53CLI5RV2ZO3XZJKX4GNJVLU7XSNXNWVA, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.361086368 (510h11m46.713871466s) [noise=true , address=145.79.227.19:4078 , piece_id=HWWSE7LNIE2AEDTVHHITLF4RCKP34LE7WBQ7PYV5ZNB66NSLEAAQ, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.361463306 (510h11m46.714248414s) [noise=true , address=145.79.227.18:4006 , piece_id=IU4SOF24YRAFJSEBXB4Y77WAGZIPBXSDAJHSV65Q2BETGFG3JN7Q, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.362015862 (510h11m46.71480097s) [noise=true , address=145.79.227.19:4079 , piece_id=A7K57FKGAZM4TFBWDXTQ5OOJKKQNEX4V3Z7TIBCTD33TEGMKISDA, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.363147756 (510h11m46.715932864s) [noise=true , address=145.79.227.5:4223 , piece_id=76NGCGRCKNNO5NXQHLRWR4E6BGMJ4PRZF6GWJTI3JU3BOQCMT7RQ, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.363521839 (510h11m46.716306937s) [noise=true , address=145.79.227.15:4875 , piece_id=5NWUJI6LA67E3BMPJKYC6JOX6RFJKLOGR5CZCFOLJG3G343NI63Q, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.377481134 (510h11m46.730266242s) [noise=true , address=145.79.227.4:4185 , piece_id=2I6ZXQ6ROSRCTKFOJDIIZXLWC7CFUDE2R7KQXC5IQLUCPZP32Y6Q, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.377709905 (510h11m46.730495003s) [noise=true , address=145.79.227.4:4186 , piece_id=R6CTLQAW5GI37OXFNQDCC4TNBFFU6ZANXJSS3XYWFSMW6MJDSQFQ, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:11.395345129 (510h11m46.748130237s) [noise=true , address=145.79.227.11:4731 , piece_id=JIGUPCONJ345Y53ITWB3PPRGA6D4ARC4VUGLGFTS55DEBLJI5K5A, seg_pos={0 0}, upload=449108]: Done uploading piece. err: <nil> Sep 19 20:20:41.246688013 (510h12m16.599473111s) [noise=true , address=77.90.31.16:4735 , piece_id=TJYEFSUKL4BT3G3FSQAW3JUOSANB33WVVLQ54YP42V5MM3C3XLAA, seg_pos={0 0}, upload=449108]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: write tcp 5.161.89.252:11902->77.90.31.16:4735: use of closed network connection; write tcp 5.161.89.252:11902->77.90.31.16:4735: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:11902->77.90.31.16:4735: use of closed network connection Sep 19 20:20:41.246747925 (510h12m16.599533024s) [noise=true , address=77.90.31.15:4698 , piece_id=Z6JMNN7ZFUWRQQTPPKRY4SGSU7UBXGMG6QQWLRMUE4QIM5Q4NXRQ, seg_pos={0 0}, upload=449108]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 5.161.89.252:45752->77.90.31.15:4698: use of closed network connection; write tcp 5.161.89.252:45752->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:45752->77.90.31.15:4698: use of closed network connection Sep 19 20:20:41.246781178 (510h12m16.599566275s) [noise=true , address=45.91.116.10:4083 , piece_id=RSCNBHOJHMZCYFL45UMUWMFAYHGFNHVRNOM7LKFRBTMUZKLTPANA, seg_pos={0 0}, upload=449108]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 5.161.89.252:12674->45.91.116.10:4083: use of closed network connection; write tcp 5.161.89.252:12674->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:12674->45.91.116.10:4083: use of closed network connection Sep 19 20:20:41.246826683 (510h12m16.599611811s) [noise=true , address=77.90.31.13:4628 , piece_id=P46LZD2I34SYMTZ233334N6HGT3TCKU2HXDHK5YBHQPR6H37S7DQ, seg_pos={0 0}, upload=449108]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 5.161.89.252:40710->77.90.31.13:4628: use of closed network connection; write tcp 5.161.89.252:40710->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:40710->77.90.31.13:4628: use of closed network connection Sep 19 20:20:41.246846630 (510h12m16.599631739s) [noise=true , address=45.91.116.5:4297 , piece_id=GMT43VQ4DSLKZB52LCEZ4M4NQLTTXABQF2HYKI7ZL2MV5RJ5MY4Q, seg_pos={0 0}, upload=449108]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: write tcp 5.161.89.252:30614->45.91.116.5:4297: use of closed network connection; write tcp 5.161.89.252:30614->45.91.116.5:4297: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:30614->45.91.116.5:4297: use of closed network connection Sep 19 20:20:41.246865556 (510h12m16.599650654s) [seg_pos={0 0}, upload=449108]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 19 20:20:41.264333648 (510h12m16.617118775s) [upload=449108]: Done uploading segments... Sep 19 20:20:41.264352643 (510h12m16.617137781s) [upload=449108]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.019560053s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..............................................XX.XXXX.XXX..X] duration: 30.000762272s FAIL (2025-09-19 20:20:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000779324s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...............................................XX.XXXX.XXX..] duration: 725.62777ms FAIL (2025-09-19 20:19:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 725.646325ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.............................................X..X.XXXXXX.X..] duration: 481.000655ms FAIL (2025-09-19 20:19:01 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 20:16:48.538156659 (510h8m23.890941758s) [upload=449061]: Starting upload Sep 19 20:16:48.538204108 (510h8m23.890989207s) [upload=449061]: Done starting upload Sep 19 20:16:48.538249184 (510h8m23.891034292s) [upload=449061]: Uploading segments... Sep 19 20:16:48.538422990 (510h8m23.891208108s) [upload=449061]: Got next segment. Inline: false Sep 19 20:16:48.538466091 (510h8m23.891251219s) [upload=449061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 20:16:48.538473014 (510h8m23.891258122s) [upload=449061]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:16:48.560086875 (510h8m23.912871983s) [seg_pos={0 0}, upload=449061]: Begin upload segment... Sep 19 20:16:48.560136528 (510h8m23.912921656s) [seg_pos={0 0}, upload=449061]: Done begin upload segment. Sep 19 20:16:48.560163388 (510h8m23.912948496s) [noise=true , address=145.79.227.7:4295 , piece_id=EEKFGO5TNHOBCOCQEV3Q3BYKDVIC4WAVYIL6FN4VABQY4DJVKNYA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560169059 (510h8m23.912954157s) [noise=true , address=77.90.31.16:4733 , piece_id=WRU4CZSWK7KDNC7AH4IGTURPZXK4JUWWAFPG6OI6ELMNHGLOLFHQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560433054 (510h8m23.913218152s) [noise=true , address=45.91.116.11:4119 , piece_id=ZHQB7F7UJRD5SANSLJTM7QX4WU2J3KHP5AEEAYHR2ZS2QKMAK4EQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560521740 (510h8m23.913306868s) [noise=true , address=77.90.31.17:4770 , piece_id=TMKHWDD2HYYJNVVUEUBB2MGQU56JUPKCHJ2NCHMRF3FF7PPUXKVA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560625956 (510h8m23.913411054s) [noise=true , address=45.91.116.8:4047 , piece_id=C4V2J2P5FF6OT6R5IORDQH5BJ5SKXDQKFOFQ22DOOLKY6QCUFQ3Q, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560883911 (510h8m23.913669019s) [noise=true , address=145.79.227.19:4077 , piece_id=K75R236EZXLK2T7EPQVJJFPZ5WOBHCO7XT7O4PCUD6JMKSV7VVSQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.561208179 (510h8m23.913993287s) [noise=true , address=77.90.31.15:4699 , piece_id=HDB7OH4MONL26D3BWQ5VFQHSLMYSMMQUV2GFDB3LTYJIYSTNLLPA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.561572824 (510h8m23.914357932s) [noise=true , address=145.79.227.17:4970 , piece_id=7IHQPVXKIPMFGDRGAHVYA74JCQKG5LCXWUJAUILS3WRAEJIFDIAQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.561950983 (510h8m23.914736081s) [noise=true , address=145.79.227.18:4007 , piece_id=7C7RGRXGD4L7ZBZS73ABS7PH3NGQSHCNNZZMTIMAMOAGHKXXVM6Q, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.562295950 (510h8m23.915081058s) [noise=true , address=45.91.116.10:4082 , piece_id=XCTIW6DKGTUKTMJBGTOYODHM3X5PEH64KZYVFPKR2WCCKJYSU5OA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.562655185 (510h8m23.915440293s) [noise=true , address=45.91.116.6:4975 , piece_id=YDOWVMCG67MRUH6FD55V45NLR3MD4Q77KRG44PEQNRQHHBEKAQVQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.563004951 (510h8m23.915790059s) [noise=true , address=77.90.31.19:4842 , piece_id=Y3AWMVGMFAET5DTJNW52P7CK7U6UKYLHXHHEDDKSCZXWJPSESDPA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.563352172 (510h8m23.91613728s) [noise=true , address=45.91.116.13:4154 , piece_id=WAKR6BXL4NF7YOVRXLNEQF3LX26DASBCDL6FO5BWXFR5T53JMVLA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.563707389 (510h8m23.916492497s) [noise=true , address=77.90.31.12:4590 , piece_id=ASFZRIH7OCQEG47IUS4CYOOKVK4VMKZDMRIXV2KIA64R6HRFAHKA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.564076902 (510h8m23.91686201s) [noise=true , address=77.90.31.14:4661 , piece_id=NW2ZYHL2K4PLH47Y2W7E7WK4DOPHNWFZN2JYH4ZW6TEXZ6QUOQMA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.564415769 (510h8m23.917200877s) [noise=true , address=145.79.227.11:4730 , piece_id=NRSARPUPGNUDBL3TS5EN45WSBHJPHVYR7EED2S55FCT2SBKUSHPA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.631689526 (510h8m23.984474624s) [upload=449061]: Next returned nil segment Sep 19 20:16:48.631706167 (510h8m23.984491266s) [upload=449061]: Waiting for error group managing segments... Sep 19 20:16:48.640411535 (510h8m23.993196632s) [noise=true , address=145.79.227.18:4007 , piece_id=7C7RGRXGD4L7ZBZS73ABS7PH3NGQSHCNNZZMTIMAMOAGHKXXVM6Q, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.784051784 (510h8m24.136836912s) [noise=true , address=145.79.227.19:4077 , piece_id=K75R236EZXLK2T7EPQVJJFPZ5WOBHCO7XT7O4PCUD6JMKSV7VVSQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.943366246 (510h8m24.296151375s) [noise=true , address=145.79.227.11:4730 , piece_id=NRSARPUPGNUDBL3TS5EN45WSBHJPHVYR7EED2S55FCT2SBKUSHPA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.945744129 (510h8m24.298529267s) [noise=true , address=145.79.227.17:4970 , piece_id=7IHQPVXKIPMFGDRGAHVYA74JCQKG5LCXWUJAUILS3WRAEJIFDIAQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.957462660 (510h8m24.310247789s) [noise=true , address=145.79.227.7:4295 , piece_id=EEKFGO5TNHOBCOCQEV3Q3BYKDVIC4WAVYIL6FN4VABQY4DJVKNYA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:07.152003889 (510h8m42.504789007s) [noise=true , address=45.91.116.6:4975 , piece_id=YDOWVMCG67MRUH6FD55V45NLR3MD4Q77KRG44PEQNRQHHBEKAQVQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:07.399660267 (510h8m42.752445395s) [noise=true , address=77.90.31.14:4661 , piece_id=NW2ZYHL2K4PLH47Y2W7E7WK4DOPHNWFZN2JYH4ZW6TEXZ6QUOQMA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:07.757832880 (510h8m43.110618009s) [noise=true , address=45.91.116.10:4082 , piece_id=XCTIW6DKGTUKTMJBGTOYODHM3X5PEH64KZYVFPKR2WCCKJYSU5OA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:08.550025885 (510h8m43.902811014s) [noise=true , address=45.91.116.13:4154 , piece_id=WAKR6BXL4NF7YOVRXLNEQF3LX26DASBCDL6FO5BWXFR5T53JMVLA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:09.302353190 (510h8m44.655138288s) [noise=true , address=45.91.116.11:4119 , piece_id=ZHQB7F7UJRD5SANSLJTM7QX4WU2J3KHP5AEEAYHR2ZS2QKMAK4EQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:11.096365302 (510h8m46.449150439s) [noise=true , address=77.90.31.19:4842 , piece_id=Y3AWMVGMFAET5DTJNW52P7CK7U6UKYLHXHHEDDKSCZXWJPSESDPA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:11.568629692 (510h8m46.9214148s) [noise=true , address=77.90.31.16:4733 , piece_id=WRU4CZSWK7KDNC7AH4IGTURPZXK4JUWWAFPG6OI6ELMNHGLOLFHQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:12.293660622 (510h8m47.646445749s) [noise=true , address=77.90.31.15:4699 , piece_id=HDB7OH4MONL26D3BWQ5VFQHSLMYSMMQUV2GFDB3LTYJIYSTNLLPA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:12.293695777 (510h8m47.646480906s) [seg_pos={0 0}, upload=449061]: Segment reached optimal threshold of 13 pieces. Sep 19 20:17:12.293994468 (510h8m47.646779607s) [noise=true , address=77.90.31.17:4770 , piece_id=TMKHWDD2HYYJNVVUEUBB2MGQU56JUPKCHJ2NCHMRF3FF7PPUXKVA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 5.161.89.252:15460->77.90.31.17:4770: use of closed network connection; write tcp 5.161.89.252:15460->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15460->77.90.31.17:4770: use of closed network connection Sep 19 20:17:12.294009857 (510h8m47.646794986s) [noise=true , address=45.91.116.8:4047 , piece_id=C4V2J2P5FF6OT6R5IORDQH5BJ5SKXDQKFOFQ22DOOLKY6QCUFQ3Q, seg_pos={0 0}, upload=449061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 5.161.89.252:61808->45.91.116.8:4047: use of closed network connection; write tcp 5.161.89.252:61808->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61808->45.91.116.8:4047: use of closed network connection Sep 19 20:17:12.294057958 (510h8m47.646843086s) [noise=true , address=77.90.31.12:4590 , piece_id=ASFZRIH7OCQEG47IUS4CYOOKVK4VMKZDMRIXV2KIA64R6HRFAHKA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: write tcp 5.161.89.252:31122->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:31122->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:31122->77.90.31.12:4590: use of closed network connection Sep 19 20:17:12.294094366 (510h8m47.646879495s) [seg_pos={0 0}, upload=449061]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 20:17:12.294173936 (510h8m47.646959063s) [upload=449061]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 20:17:12.294179466 (510h8m47.646964594s) [upload=449061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 20:17:12.352729843 (510h8m47.705514972s) [upload=449061]: Done uploading segments... Sep 19 20:17:12.352747947 (510h8m47.705533075s) [upload=449061]: Upload finished. err: <nil> Duration: 481.021375ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [................................................XX.XXXX.XXX.] duration: 654.959804ms FAIL (2025-09-19 20:18:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 654.972558ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................................................XX.XXXX.XXX] duration: 30.001227633s FAIL (2025-09-19 20:18:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001240366s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..............................................X..X.XXXXXX.X.] duration: 15.962771554s FAIL (2025-09-19 20:18:18 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 20:16:48.538156659 (510h8m23.890941758s) [upload=449061]: Starting upload Sep 19 20:16:48.538204108 (510h8m23.890989207s) [upload=449061]: Done starting upload Sep 19 20:16:48.538249184 (510h8m23.891034292s) [upload=449061]: Uploading segments... Sep 19 20:16:48.538422990 (510h8m23.891208108s) [upload=449061]: Got next segment. Inline: false Sep 19 20:16:48.538466091 (510h8m23.891251219s) [upload=449061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 20:16:48.538473014 (510h8m23.891258122s) [upload=449061]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:16:48.560086875 (510h8m23.912871983s) [seg_pos={0 0}, upload=449061]: Begin upload segment... Sep 19 20:16:48.560136528 (510h8m23.912921656s) [seg_pos={0 0}, upload=449061]: Done begin upload segment. Sep 19 20:16:48.560163388 (510h8m23.912948496s) [noise=true , address=145.79.227.7:4295 , piece_id=EEKFGO5TNHOBCOCQEV3Q3BYKDVIC4WAVYIL6FN4VABQY4DJVKNYA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560169059 (510h8m23.912954157s) [noise=true , address=77.90.31.16:4733 , piece_id=WRU4CZSWK7KDNC7AH4IGTURPZXK4JUWWAFPG6OI6ELMNHGLOLFHQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560433054 (510h8m23.913218152s) [noise=true , address=45.91.116.11:4119 , piece_id=ZHQB7F7UJRD5SANSLJTM7QX4WU2J3KHP5AEEAYHR2ZS2QKMAK4EQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560521740 (510h8m23.913306868s) [noise=true , address=77.90.31.17:4770 , piece_id=TMKHWDD2HYYJNVVUEUBB2MGQU56JUPKCHJ2NCHMRF3FF7PPUXKVA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560625956 (510h8m23.913411054s) [noise=true , address=45.91.116.8:4047 , piece_id=C4V2J2P5FF6OT6R5IORDQH5BJ5SKXDQKFOFQ22DOOLKY6QCUFQ3Q, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560883911 (510h8m23.913669019s) [noise=true , address=145.79.227.19:4077 , piece_id=K75R236EZXLK2T7EPQVJJFPZ5WOBHCO7XT7O4PCUD6JMKSV7VVSQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.561208179 (510h8m23.913993287s) [noise=true , address=77.90.31.15:4699 , piece_id=HDB7OH4MONL26D3BWQ5VFQHSLMYSMMQUV2GFDB3LTYJIYSTNLLPA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.561572824 (510h8m23.914357932s) [noise=true , address=145.79.227.17:4970 , piece_id=7IHQPVXKIPMFGDRGAHVYA74JCQKG5LCXWUJAUILS3WRAEJIFDIAQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.561950983 (510h8m23.914736081s) [noise=true , address=145.79.227.18:4007 , piece_id=7C7RGRXGD4L7ZBZS73ABS7PH3NGQSHCNNZZMTIMAMOAGHKXXVM6Q, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.562295950 (510h8m23.915081058s) [noise=true , address=45.91.116.10:4082 , piece_id=XCTIW6DKGTUKTMJBGTOYODHM3X5PEH64KZYVFPKR2WCCKJYSU5OA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.562655185 (510h8m23.915440293s) [noise=true , address=45.91.116.6:4975 , piece_id=YDOWVMCG67MRUH6FD55V45NLR3MD4Q77KRG44PEQNRQHHBEKAQVQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.563004951 (510h8m23.915790059s) [noise=true , address=77.90.31.19:4842 , piece_id=Y3AWMVGMFAET5DTJNW52P7CK7U6UKYLHXHHEDDKSCZXWJPSESDPA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.563352172 (510h8m23.91613728s) [noise=true , address=45.91.116.13:4154 , piece_id=WAKR6BXL4NF7YOVRXLNEQF3LX26DASBCDL6FO5BWXFR5T53JMVLA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.563707389 (510h8m23.916492497s) [noise=true , address=77.90.31.12:4590 , piece_id=ASFZRIH7OCQEG47IUS4CYOOKVK4VMKZDMRIXV2KIA64R6HRFAHKA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.564076902 (510h8m23.91686201s) [noise=true , address=77.90.31.14:4661 , piece_id=NW2ZYHL2K4PLH47Y2W7E7WK4DOPHNWFZN2JYH4ZW6TEXZ6QUOQMA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.564415769 (510h8m23.917200877s) [noise=true , address=145.79.227.11:4730 , piece_id=NRSARPUPGNUDBL3TS5EN45WSBHJPHVYR7EED2S55FCT2SBKUSHPA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.631689526 (510h8m23.984474624s) [upload=449061]: Next returned nil segment Sep 19 20:16:48.631706167 (510h8m23.984491266s) [upload=449061]: Waiting for error group managing segments... Sep 19 20:16:48.640411535 (510h8m23.993196632s) [noise=true , address=145.79.227.18:4007 , piece_id=7C7RGRXGD4L7ZBZS73ABS7PH3NGQSHCNNZZMTIMAMOAGHKXXVM6Q, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.784051784 (510h8m24.136836912s) [noise=true , address=145.79.227.19:4077 , piece_id=K75R236EZXLK2T7EPQVJJFPZ5WOBHCO7XT7O4PCUD6JMKSV7VVSQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.943366246 (510h8m24.296151375s) [noise=true , address=145.79.227.11:4730 , piece_id=NRSARPUPGNUDBL3TS5EN45WSBHJPHVYR7EED2S55FCT2SBKUSHPA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.945744129 (510h8m24.298529267s) [noise=true , address=145.79.227.17:4970 , piece_id=7IHQPVXKIPMFGDRGAHVYA74JCQKG5LCXWUJAUILS3WRAEJIFDIAQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.957462660 (510h8m24.310247789s) [noise=true , address=145.79.227.7:4295 , piece_id=EEKFGO5TNHOBCOCQEV3Q3BYKDVIC4WAVYIL6FN4VABQY4DJVKNYA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:07.152003889 (510h8m42.504789007s) [noise=true , address=45.91.116.6:4975 , piece_id=YDOWVMCG67MRUH6FD55V45NLR3MD4Q77KRG44PEQNRQHHBEKAQVQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:07.399660267 (510h8m42.752445395s) [noise=true , address=77.90.31.14:4661 , piece_id=NW2ZYHL2K4PLH47Y2W7E7WK4DOPHNWFZN2JYH4ZW6TEXZ6QUOQMA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:07.757832880 (510h8m43.110618009s) [noise=true , address=45.91.116.10:4082 , piece_id=XCTIW6DKGTUKTMJBGTOYODHM3X5PEH64KZYVFPKR2WCCKJYSU5OA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:08.550025885 (510h8m43.902811014s) [noise=true , address=45.91.116.13:4154 , piece_id=WAKR6BXL4NF7YOVRXLNEQF3LX26DASBCDL6FO5BWXFR5T53JMVLA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:09.302353190 (510h8m44.655138288s) [noise=true , address=45.91.116.11:4119 , piece_id=ZHQB7F7UJRD5SANSLJTM7QX4WU2J3KHP5AEEAYHR2ZS2QKMAK4EQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:11.096365302 (510h8m46.449150439s) [noise=true , address=77.90.31.19:4842 , piece_id=Y3AWMVGMFAET5DTJNW52P7CK7U6UKYLHXHHEDDKSCZXWJPSESDPA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:11.568629692 (510h8m46.9214148s) [noise=true , address=77.90.31.16:4733 , piece_id=WRU4CZSWK7KDNC7AH4IGTURPZXK4JUWWAFPG6OI6ELMNHGLOLFHQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:12.293660622 (510h8m47.646445749s) [noise=true , address=77.90.31.15:4699 , piece_id=HDB7OH4MONL26D3BWQ5VFQHSLMYSMMQUV2GFDB3LTYJIYSTNLLPA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:12.293695777 (510h8m47.646480906s) [seg_pos={0 0}, upload=449061]: Segment reached optimal threshold of 13 pieces. Sep 19 20:17:12.293994468 (510h8m47.646779607s) [noise=true , address=77.90.31.17:4770 , piece_id=TMKHWDD2HYYJNVVUEUBB2MGQU56JUPKCHJ2NCHMRF3FF7PPUXKVA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 5.161.89.252:15460->77.90.31.17:4770: use of closed network connection; write tcp 5.161.89.252:15460->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15460->77.90.31.17:4770: use of closed network connection Sep 19 20:17:12.294009857 (510h8m47.646794986s) [noise=true , address=45.91.116.8:4047 , piece_id=C4V2J2P5FF6OT6R5IORDQH5BJ5SKXDQKFOFQ22DOOLKY6QCUFQ3Q, seg_pos={0 0}, upload=449061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 5.161.89.252:61808->45.91.116.8:4047: use of closed network connection; write tcp 5.161.89.252:61808->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61808->45.91.116.8:4047: use of closed network connection Sep 19 20:17:12.294057958 (510h8m47.646843086s) [noise=true , address=77.90.31.12:4590 , piece_id=ASFZRIH7OCQEG47IUS4CYOOKVK4VMKZDMRIXV2KIA64R6HRFAHKA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: write tcp 5.161.89.252:31122->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:31122->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:31122->77.90.31.12:4590: use of closed network connection Sep 19 20:17:12.294094366 (510h8m47.646879495s) [seg_pos={0 0}, upload=449061]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 20:17:12.294173936 (510h8m47.646959063s) [upload=449061]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 20:17:12.294179466 (510h8m47.646964594s) [upload=449061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 20:17:12.352729843 (510h8m47.705514972s) [upload=449061]: Done uploading segments... Sep 19 20:17:12.352747947 (510h8m47.705533075s) [upload=449061]: Upload finished. err: <nil> Duration: 15.962792162s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..................................................XX.XXXX.XX] duration: 30.000839831s FAIL (2025-09-19 20:17:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000863285s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...............................................X..X.XXXXXX.X] duration: 30.003852932s FAIL (2025-09-19 20:17:18 UTC) operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 19 20:16:48.538156659 (510h8m23.890941758s) [upload=449061]: Starting upload Sep 19 20:16:48.538204108 (510h8m23.890989207s) [upload=449061]: Done starting upload Sep 19 20:16:48.538249184 (510h8m23.891034292s) [upload=449061]: Uploading segments... Sep 19 20:16:48.538422990 (510h8m23.891208108s) [upload=449061]: Got next segment. Inline: false Sep 19 20:16:48.538466091 (510h8m23.891251219s) [upload=449061]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 20:16:48.538473014 (510h8m23.891258122s) [upload=449061]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:16:48.560086875 (510h8m23.912871983s) [seg_pos={0 0}, upload=449061]: Begin upload segment... Sep 19 20:16:48.560136528 (510h8m23.912921656s) [seg_pos={0 0}, upload=449061]: Done begin upload segment. Sep 19 20:16:48.560163388 (510h8m23.912948496s) [noise=true , address=145.79.227.7:4295 , piece_id=EEKFGO5TNHOBCOCQEV3Q3BYKDVIC4WAVYIL6FN4VABQY4DJVKNYA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560169059 (510h8m23.912954157s) [noise=true , address=77.90.31.16:4733 , piece_id=WRU4CZSWK7KDNC7AH4IGTURPZXK4JUWWAFPG6OI6ELMNHGLOLFHQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560433054 (510h8m23.913218152s) [noise=true , address=45.91.116.11:4119 , piece_id=ZHQB7F7UJRD5SANSLJTM7QX4WU2J3KHP5AEEAYHR2ZS2QKMAK4EQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560521740 (510h8m23.913306868s) [noise=true , address=77.90.31.17:4770 , piece_id=TMKHWDD2HYYJNVVUEUBB2MGQU56JUPKCHJ2NCHMRF3FF7PPUXKVA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560625956 (510h8m23.913411054s) [noise=true , address=45.91.116.8:4047 , piece_id=C4V2J2P5FF6OT6R5IORDQH5BJ5SKXDQKFOFQ22DOOLKY6QCUFQ3Q, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.560883911 (510h8m23.913669019s) [noise=true , address=145.79.227.19:4077 , piece_id=K75R236EZXLK2T7EPQVJJFPZ5WOBHCO7XT7O4PCUD6JMKSV7VVSQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.561208179 (510h8m23.913993287s) [noise=true , address=77.90.31.15:4699 , piece_id=HDB7OH4MONL26D3BWQ5VFQHSLMYSMMQUV2GFDB3LTYJIYSTNLLPA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.561572824 (510h8m23.914357932s) [noise=true , address=145.79.227.17:4970 , piece_id=7IHQPVXKIPMFGDRGAHVYA74JCQKG5LCXWUJAUILS3WRAEJIFDIAQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.561950983 (510h8m23.914736081s) [noise=true , address=145.79.227.18:4007 , piece_id=7C7RGRXGD4L7ZBZS73ABS7PH3NGQSHCNNZZMTIMAMOAGHKXXVM6Q, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.562295950 (510h8m23.915081058s) [noise=true , address=45.91.116.10:4082 , piece_id=XCTIW6DKGTUKTMJBGTOYODHM3X5PEH64KZYVFPKR2WCCKJYSU5OA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.562655185 (510h8m23.915440293s) [noise=true , address=45.91.116.6:4975 , piece_id=YDOWVMCG67MRUH6FD55V45NLR3MD4Q77KRG44PEQNRQHHBEKAQVQ, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.563004951 (510h8m23.915790059s) [noise=true , address=77.90.31.19:4842 , piece_id=Y3AWMVGMFAET5DTJNW52P7CK7U6UKYLHXHHEDDKSCZXWJPSESDPA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.563352172 (510h8m23.91613728s) [noise=true , address=45.91.116.13:4154 , piece_id=WAKR6BXL4NF7YOVRXLNEQF3LX26DASBCDL6FO5BWXFR5T53JMVLA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.563707389 (510h8m23.916492497s) [noise=true , address=77.90.31.12:4590 , piece_id=ASFZRIH7OCQEG47IUS4CYOOKVK4VMKZDMRIXV2KIA64R6HRFAHKA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.564076902 (510h8m23.91686201s) [noise=true , address=77.90.31.14:4661 , piece_id=NW2ZYHL2K4PLH47Y2W7E7WK4DOPHNWFZN2JYH4ZW6TEXZ6QUOQMA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.564415769 (510h8m23.917200877s) [noise=true , address=145.79.227.11:4730 , piece_id=NRSARPUPGNUDBL3TS5EN45WSBHJPHVYR7EED2S55FCT2SBKUSHPA, seg_pos={0 0}, upload=449061]: Uploading piece... Sep 19 20:16:48.631689526 (510h8m23.984474624s) [upload=449061]: Next returned nil segment Sep 19 20:16:48.631706167 (510h8m23.984491266s) [upload=449061]: Waiting for error group managing segments... Sep 19 20:16:48.640411535 (510h8m23.993196632s) [noise=true , address=145.79.227.18:4007 , piece_id=7C7RGRXGD4L7ZBZS73ABS7PH3NGQSHCNNZZMTIMAMOAGHKXXVM6Q, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.784051784 (510h8m24.136836912s) [noise=true , address=145.79.227.19:4077 , piece_id=K75R236EZXLK2T7EPQVJJFPZ5WOBHCO7XT7O4PCUD6JMKSV7VVSQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.943366246 (510h8m24.296151375s) [noise=true , address=145.79.227.11:4730 , piece_id=NRSARPUPGNUDBL3TS5EN45WSBHJPHVYR7EED2S55FCT2SBKUSHPA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.945744129 (510h8m24.298529267s) [noise=true , address=145.79.227.17:4970 , piece_id=7IHQPVXKIPMFGDRGAHVYA74JCQKG5LCXWUJAUILS3WRAEJIFDIAQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:16:48.957462660 (510h8m24.310247789s) [noise=true , address=145.79.227.7:4295 , piece_id=EEKFGO5TNHOBCOCQEV3Q3BYKDVIC4WAVYIL6FN4VABQY4DJVKNYA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:07.152003889 (510h8m42.504789007s) [noise=true , address=45.91.116.6:4975 , piece_id=YDOWVMCG67MRUH6FD55V45NLR3MD4Q77KRG44PEQNRQHHBEKAQVQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:07.399660267 (510h8m42.752445395s) [noise=true , address=77.90.31.14:4661 , piece_id=NW2ZYHL2K4PLH47Y2W7E7WK4DOPHNWFZN2JYH4ZW6TEXZ6QUOQMA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:07.757832880 (510h8m43.110618009s) [noise=true , address=45.91.116.10:4082 , piece_id=XCTIW6DKGTUKTMJBGTOYODHM3X5PEH64KZYVFPKR2WCCKJYSU5OA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:08.550025885 (510h8m43.902811014s) [noise=true , address=45.91.116.13:4154 , piece_id=WAKR6BXL4NF7YOVRXLNEQF3LX26DASBCDL6FO5BWXFR5T53JMVLA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:09.302353190 (510h8m44.655138288s) [noise=true , address=45.91.116.11:4119 , piece_id=ZHQB7F7UJRD5SANSLJTM7QX4WU2J3KHP5AEEAYHR2ZS2QKMAK4EQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:11.096365302 (510h8m46.449150439s) [noise=true , address=77.90.31.19:4842 , piece_id=Y3AWMVGMFAET5DTJNW52P7CK7U6UKYLHXHHEDDKSCZXWJPSESDPA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:11.568629692 (510h8m46.9214148s) [noise=true , address=77.90.31.16:4733 , piece_id=WRU4CZSWK7KDNC7AH4IGTURPZXK4JUWWAFPG6OI6ELMNHGLOLFHQ, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:12.293660622 (510h8m47.646445749s) [noise=true , address=77.90.31.15:4699 , piece_id=HDB7OH4MONL26D3BWQ5VFQHSLMYSMMQUV2GFDB3LTYJIYSTNLLPA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: <nil> Sep 19 20:17:12.293695777 (510h8m47.646480906s) [seg_pos={0 0}, upload=449061]: Segment reached optimal threshold of 13 pieces. Sep 19 20:17:12.293994468 (510h8m47.646779607s) [noise=true , address=77.90.31.17:4770 , piece_id=TMKHWDD2HYYJNVVUEUBB2MGQU56JUPKCHJ2NCHMRF3FF7PPUXKVA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:125ivanwRPxZhNW6ryony9FCETtFegxPRZg18B3AB2B9ch8865R): protocol: write tcp 5.161.89.252:15460->77.90.31.17:4770: use of closed network connection; write tcp 5.161.89.252:15460->77.90.31.17:4770: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:15460->77.90.31.17:4770: use of closed network connection Sep 19 20:17:12.294009857 (510h8m47.646794986s) [noise=true , address=45.91.116.8:4047 , piece_id=C4V2J2P5FF6OT6R5IORDQH5BJ5SKXDQKFOFQ22DOOLKY6QCUFQ3Q, seg_pos={0 0}, upload=449061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 5.161.89.252:61808->45.91.116.8:4047: use of closed network connection; write tcp 5.161.89.252:61808->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61808->45.91.116.8:4047: use of closed network connection Sep 19 20:17:12.294057958 (510h8m47.646843086s) [noise=true , address=77.90.31.12:4590 , piece_id=ASFZRIH7OCQEG47IUS4CYOOKVK4VMKZDMRIXV2KIA64R6HRFAHKA, seg_pos={0 0}, upload=449061]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: write tcp 5.161.89.252:31122->77.90.31.12:4590: use of closed network connection; write tcp 5.161.89.252:31122->77.90.31.12:4590: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:31122->77.90.31.12:4590: use of closed network connection Sep 19 20:17:12.294094366 (510h8m47.646879495s) [seg_pos={0 0}, upload=449061]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 19 20:17:12.294173936 (510h8m47.646959063s) [upload=449061]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 19 20:17:12.294179466 (510h8m47.646964594s) [upload=449061]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 19 20:17:12.352729843 (510h8m47.705514972s) [upload=449061]: Done uploading segments... Sep 19 20:17:12.352747947 (510h8m47.705533075s) [upload=449061]: Upload finished. err: <nil> Duration: 30.003880594s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...................................................XX.XXXX.X] duration: 30.000951923s FAIL (2025-09-19 20:17:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000963964s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................................................XX.XXXX.] duration: 444.798648ms FAIL (2025-09-19 20:15:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 444.863519ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [................................................X..X.XXXXXX.] duration: 761.649276ms FAIL (2025-09-19 20:15:49 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ue , address=45.91.116.10:4081 , piece_id=FDZAAIB5FQ7UU4I33NZBTBTZ3Q7XIEYXLP6VC7HYUNV4TIOEFYPQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.030973081 (510h6m48.383758189s) [noise=true , address=45.91.116.7:4010 , piece_id=NBJINUMY5ZU7O3XSWQBO3XGSWIA2BRL57HLY2ZBE4OSMTSQ4CMPQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.031029827 (510h6m48.383814926s) [noise=true , address=77.90.31.15:4699 , piece_id=5TDA36LUDKFAWNC5HOWPTTKWGBPXOJM4MJVJ34UAFZ3VISXZXDQA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.031429989 (510h6m48.384215097s) [noise=true , address=77.90.31.21:4877 , piece_id=7BASA44XHUQHSM5B2Z7652CBRY4WZW7TCMZMZXMLXAWXD6Z5DKMA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.031443063 (510h6m48.384228171s) [noise=true , address=145.79.227.7:4294 , piece_id=MFBKQS5YZWPXIXS6LNLGW3HDZWE2ZKN7ERBBYVX6U27LEPBF2CHA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.031809361 (510h6m48.384594469s) [noise=true , address=45.91.116.4:4937 , piece_id=LWTE7CNKCXYA6TYWOASLDEHJZDRDIRMRDY7G7X2SKVSM5UFCHBIA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.032153406 (510h6m48.384938514s) [noise=true , address=45.91.116.9:4263 , piece_id=IYYEKX3QYG6AL3NT7WOHSS4F2L7DVQ5MEOZEKLOBEO7YZ6TOTKEA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.032498243 (510h6m48.385283351s) [noise=true , address=145.79.227.3:4149 , piece_id=4XCVXVMFICP4OIZ2IALPT36SBWMY6XCZXYZBNJJ3IJOPRUL2LAPA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.032876052 (510h6m48.38566116s) [noise=true , address=45.91.116.5:4298 , piece_id=3LL4EWUSD4IDSBJZDKOC7JDDN2KHVBMW5GL7LWQQV3GQZVCEF7TQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.033207964 (510h6m48.385993072s) [noise=true , address=45.91.116.12:4225 , piece_id=7J7FCM4IMRDPRO5SXTL5NRCMT2S7LWJLEG7O7KIXIXQ27SMHEU5A, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.033554054 (510h6m48.386339162s) [noise=true , address=77.90.31.19:4843 , piece_id=KU4BMJFJLD4WD4IV7DU7NYASN7F4SJDCSNJ3FCHNHSABM7HLROIA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.033887530 (510h6m48.386672638s) [noise=true , address=145.79.227.14:4839 , piece_id=W4XOPNY5YLJQ22YTRVYHHF665636ES644ZDGACU3VEUKLGZ6TTBQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.034239581 (510h6m48.387024689s) [noise=true , address=145.79.227.11:4729 , piece_id=Z2WKDTWMFE23BEOASGUIEUYDWIPBFEZPZO42RVOU5IG3DSUXMLQQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.035188222 (510h6m48.38797333s) [noise=true , address=145.79.227.15:4875 , piece_id=2SP2ERRBAOXC2DW5DR3IUKUFWFBLKYWFZOENGA3KTHATRFBBJ5UQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.035550471 (510h6m48.388335579s) [noise=true , address=77.90.31.12:4592 , piece_id=XYLRAM7BMAC5245KZ5OGNWJCT5HR6K6YES5CEZMMTVHXOMDRMWGQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.040692290 (510h6m48.393477408s) [noise=true , address=45.91.116.5:4297 , piece_id=ZV6EYEUXJWSZZWMSRXTBEYQKVL3N5I7LV47TCMRUMNBFNHODLSKQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.109822390 (510h6m48.462607498s) [upload=449035]: Next returned nil segment Sep 19 20:15:13.109833583 (510h6m48.462618691s) [upload=449035]: Waiting for error group managing segments... Sep 19 20:15:13.117400211 (510h6m48.47018531s) [noise=true , address=145.79.227.14:4839 , piece_id=W4XOPNY5YLJQ22YTRVYHHF665636ES644ZDGACU3VEUKLGZ6TTBQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:13.122980403 (510h6m48.475765501s) [noise=true , address=145.79.227.15:4875 , piece_id=2SP2ERRBAOXC2DW5DR3IUKUFWFBLKYWFZOENGA3KTHATRFBBJ5UQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:13.123937229 (510h6m48.476722337s) [noise=true , address=145.79.227.11:4729 , piece_id=Z2WKDTWMFE23BEOASGUIEUYDWIPBFEZPZO42RVOU5IG3DSUXMLQQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:13.125562299 (510h6m48.478347407s) [noise=true , address=145.79.227.7:4294 , piece_id=MFBKQS5YZWPXIXS6LNLGW3HDZWE2ZKN7ERBBYVX6U27LEPBF2CHA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:13.125940208 (510h6m48.478725305s) [noise=true , address=145.79.227.3:4149 , piece_id=4XCVXVMFICP4OIZ2IALPT36SBWMY6XCZXYZBNJJ3IJOPRUL2LAPA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:42.980667971 (510h7m18.333453079s) [noise=true , address=45.91.116.5:4298 , piece_id=3LL4EWUSD4IDSBJZDKOC7JDDN2KHVBMW5GL7LWQQV3GQZVCEF7TQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.980719709 (510h7m18.333504817s) [noise=true , address=45.91.116.12:4225 , piece_id=7J7FCM4IMRDPRO5SXTL5NRCMT2S7LWJLEG7O7KIXIXQ27SMHEU5A, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 5.161.89.252:30078->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.980777668 (510h7m18.333562766s) [noise=true , address=45.91.116.7:4010 , piece_id=NBJINUMY5ZU7O3XSWQBO3XGSWIA2BRL57HLY2ZBE4OSMTSQ4CMPQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.980810269 (510h7m18.333595367s) [noise=true , address=77.90.31.21:4877 , piece_id=7BASA44XHUQHSM5B2Z7652CBRY4WZW7TCMZMZXMLXAWXD6Z5DKMA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.980972664 (510h7m18.333757772s) [noise=true , address=77.90.31.15:4699 , piece_id=5TDA36LUDKFAWNC5HOWPTTKWGBPXOJM4MJVJ34UAFZ3VISXZXDQA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981114850 (510h7m18.333899958s) [noise=true , address=45.91.116.9:4263 , piece_id=IYYEKX3QYG6AL3NT7WOHSS4F2L7DVQ5MEOZEKLOBEO7YZ6TOTKEA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 5.161.89.252:28678->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981157009 (510h7m18.333942117s) [noise=true , address=45.91.116.4:4937 , piece_id=LWTE7CNKCXYA6TYWOASLDEHJZDRDIRMRDY7G7X2SKVSM5UFCHBIA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: read tcp 5.161.89.252:45046->45.91.116.4:4937: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981224225 (510h7m18.334009323s) [noise=true , address=77.90.31.12:4592 , piece_id=XYLRAM7BMAC5245KZ5OGNWJCT5HR6K6YES5CEZMMTVHXOMDRMWGQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 5.161.89.252:21460->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981232090 (510h7m18.334017188s) [noise=true , address=77.90.31.19:4843 , piece_id=KU4BMJFJLD4WD4IV7DU7NYASN7F4SJDCSNJ3FCHNHSABM7HLROIA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: read tcp 5.161.89.252:26994->77.90.31.19:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981268959 (510h7m18.334054067s) [noise=true , address=45.91.116.5:4297 , piece_id=ZV6EYEUXJWSZZWMSRXTBEYQKVL3N5I7LV47TCMRUMNBFNHODLSKQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 5.161.89.252:23864->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981275261 (510h7m18.334060359s) [noise=true , address=45.91.116.10:4081 , piece_id=FDZAAIB5FQ7UU4I33NZBTBTZ3Q7XIEYXLP6VC7HYUNV4TIOEFYPQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: rpc: dial tcp 45.91.116.10:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981297513 (510h7m18.334082611s) [seg_pos={0 0}, upload=449035]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Sep 19 20:15:43.040834362 (510h7m18.39361946s) [upload=449035]: Done uploading segments... Sep 19 20:15:43.040852987 (510h7m18.393638095s) [upload=449035]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 761.67791ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [.................................................X..X.XXXXXX] duration: 30.061089438s FAIL (2025-09-19 20:15:43 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== ue , address=45.91.116.10:4081 , piece_id=FDZAAIB5FQ7UU4I33NZBTBTZ3Q7XIEYXLP6VC7HYUNV4TIOEFYPQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.030973081 (510h6m48.383758189s) [noise=true , address=45.91.116.7:4010 , piece_id=NBJINUMY5ZU7O3XSWQBO3XGSWIA2BRL57HLY2ZBE4OSMTSQ4CMPQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.031029827 (510h6m48.383814926s) [noise=true , address=77.90.31.15:4699 , piece_id=5TDA36LUDKFAWNC5HOWPTTKWGBPXOJM4MJVJ34UAFZ3VISXZXDQA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.031429989 (510h6m48.384215097s) [noise=true , address=77.90.31.21:4877 , piece_id=7BASA44XHUQHSM5B2Z7652CBRY4WZW7TCMZMZXMLXAWXD6Z5DKMA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.031443063 (510h6m48.384228171s) [noise=true , address=145.79.227.7:4294 , piece_id=MFBKQS5YZWPXIXS6LNLGW3HDZWE2ZKN7ERBBYVX6U27LEPBF2CHA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.031809361 (510h6m48.384594469s) [noise=true , address=45.91.116.4:4937 , piece_id=LWTE7CNKCXYA6TYWOASLDEHJZDRDIRMRDY7G7X2SKVSM5UFCHBIA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.032153406 (510h6m48.384938514s) [noise=true , address=45.91.116.9:4263 , piece_id=IYYEKX3QYG6AL3NT7WOHSS4F2L7DVQ5MEOZEKLOBEO7YZ6TOTKEA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.032498243 (510h6m48.385283351s) [noise=true , address=145.79.227.3:4149 , piece_id=4XCVXVMFICP4OIZ2IALPT36SBWMY6XCZXYZBNJJ3IJOPRUL2LAPA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.032876052 (510h6m48.38566116s) [noise=true , address=45.91.116.5:4298 , piece_id=3LL4EWUSD4IDSBJZDKOC7JDDN2KHVBMW5GL7LWQQV3GQZVCEF7TQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.033207964 (510h6m48.385993072s) [noise=true , address=45.91.116.12:4225 , piece_id=7J7FCM4IMRDPRO5SXTL5NRCMT2S7LWJLEG7O7KIXIXQ27SMHEU5A, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.033554054 (510h6m48.386339162s) [noise=true , address=77.90.31.19:4843 , piece_id=KU4BMJFJLD4WD4IV7DU7NYASN7F4SJDCSNJ3FCHNHSABM7HLROIA, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.033887530 (510h6m48.386672638s) [noise=true , address=145.79.227.14:4839 , piece_id=W4XOPNY5YLJQ22YTRVYHHF665636ES644ZDGACU3VEUKLGZ6TTBQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.034239581 (510h6m48.387024689s) [noise=true , address=145.79.227.11:4729 , piece_id=Z2WKDTWMFE23BEOASGUIEUYDWIPBFEZPZO42RVOU5IG3DSUXMLQQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.035188222 (510h6m48.38797333s) [noise=true , address=145.79.227.15:4875 , piece_id=2SP2ERRBAOXC2DW5DR3IUKUFWFBLKYWFZOENGA3KTHATRFBBJ5UQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.035550471 (510h6m48.388335579s) [noise=true , address=77.90.31.12:4592 , piece_id=XYLRAM7BMAC5245KZ5OGNWJCT5HR6K6YES5CEZMMTVHXOMDRMWGQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.040692290 (510h6m48.393477408s) [noise=true , address=45.91.116.5:4297 , piece_id=ZV6EYEUXJWSZZWMSRXTBEYQKVL3N5I7LV47TCMRUMNBFNHODLSKQ, seg_pos={0 0}, upload=449035]: Uploading piece... Sep 19 20:15:13.109822390 (510h6m48.462607498s) [upload=449035]: Next returned nil segment Sep 19 20:15:13.109833583 (510h6m48.462618691s) [upload=449035]: Waiting for error group managing segments... Sep 19 20:15:13.117400211 (510h6m48.47018531s) [noise=true , address=145.79.227.14:4839 , piece_id=W4XOPNY5YLJQ22YTRVYHHF665636ES644ZDGACU3VEUKLGZ6TTBQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:13.122980403 (510h6m48.475765501s) [noise=true , address=145.79.227.15:4875 , piece_id=2SP2ERRBAOXC2DW5DR3IUKUFWFBLKYWFZOENGA3KTHATRFBBJ5UQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:13.123937229 (510h6m48.476722337s) [noise=true , address=145.79.227.11:4729 , piece_id=Z2WKDTWMFE23BEOASGUIEUYDWIPBFEZPZO42RVOU5IG3DSUXMLQQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:13.125562299 (510h6m48.478347407s) [noise=true , address=145.79.227.7:4294 , piece_id=MFBKQS5YZWPXIXS6LNLGW3HDZWE2ZKN7ERBBYVX6U27LEPBF2CHA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:13.125940208 (510h6m48.478725305s) [noise=true , address=145.79.227.3:4149 , piece_id=4XCVXVMFICP4OIZ2IALPT36SBWMY6XCZXYZBNJJ3IJOPRUL2LAPA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: <nil> Sep 19 20:15:42.980667971 (510h7m18.333453079s) [noise=true , address=45.91.116.5:4298 , piece_id=3LL4EWUSD4IDSBJZDKOC7JDDN2KHVBMW5GL7LWQQV3GQZVCEF7TQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.980719709 (510h7m18.333504817s) [noise=true , address=45.91.116.12:4225 , piece_id=7J7FCM4IMRDPRO5SXTL5NRCMT2S7LWJLEG7O7KIXIXQ27SMHEU5A, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: read tcp 5.161.89.252:30078->45.91.116.12:4225: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.980777668 (510h7m18.333562766s) [noise=true , address=45.91.116.7:4010 , piece_id=NBJINUMY5ZU7O3XSWQBO3XGSWIA2BRL57HLY2ZBE4OSMTSQ4CMPQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.980810269 (510h7m18.333595367s) [noise=true , address=77.90.31.21:4877 , piece_id=7BASA44XHUQHSM5B2Z7652CBRY4WZW7TCMZMZXMLXAWXD6Z5DKMA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.980972664 (510h7m18.333757772s) [noise=true , address=77.90.31.15:4699 , piece_id=5TDA36LUDKFAWNC5HOWPTTKWGBPXOJM4MJVJ34UAFZ3VISXZXDQA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981114850 (510h7m18.333899958s) [noise=true , address=45.91.116.9:4263 , piece_id=IYYEKX3QYG6AL3NT7WOHSS4F2L7DVQ5MEOZEKLOBEO7YZ6TOTKEA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 5.161.89.252:28678->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981157009 (510h7m18.333942117s) [noise=true , address=45.91.116.4:4937 , piece_id=LWTE7CNKCXYA6TYWOASLDEHJZDRDIRMRDY7G7X2SKVSM5UFCHBIA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq, address:45.91.116.4:4937): protocol: read tcp 5.161.89.252:45046->45.91.116.4:4937: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981224225 (510h7m18.334009323s) [noise=true , address=77.90.31.12:4592 , piece_id=XYLRAM7BMAC5245KZ5OGNWJCT5HR6K6YES5CEZMMTVHXOMDRMWGQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: read tcp 5.161.89.252:21460->77.90.31.12:4592: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981232090 (510h7m18.334017188s) [noise=true , address=77.90.31.19:4843 , piece_id=KU4BMJFJLD4WD4IV7DU7NYASN7F4SJDCSNJ3FCHNHSABM7HLROIA, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: read tcp 5.161.89.252:26994->77.90.31.19:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981268959 (510h7m18.334054067s) [noise=true , address=45.91.116.5:4297 , piece_id=ZV6EYEUXJWSZZWMSRXTBEYQKVL3N5I7LV47TCMRUMNBFNHODLSKQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 5.161.89.252:23864->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981275261 (510h7m18.334060359s) [noise=true , address=45.91.116.10:4081 , piece_id=FDZAAIB5FQ7UU4I33NZBTBTZ3Q7XIEYXLP6VC7HYUNV4TIOEFYPQ, seg_pos={0 0}, upload=449035]: Done uploading piece. err: ecclient: upload failed (node:12UqimG6nUH8jMQVyovyRz8PXqRzRGbFF2u1UjScB5RdsvHECqG, address:45.91.116.10:4081): protocol: rpc: dial tcp 45.91.116.10:4081: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:15:42.981297513 (510h7m18.334082611s) [seg_pos={0 0}, upload=449035]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Sep 19 20:15:43.040834362 (510h7m18.39361946s) [upload=449035]: Done uploading segments... Sep 19 20:15:43.040852987 (510h7m18.393638095s) [upload=449035]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 30.061104736s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................................XX.XXXX] duration: 30.001195864s FAIL (2025-09-19 20:15:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001210041s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......................................................XX.XXX] duration: 30.000582631s FAIL (2025-09-19 20:14:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.00059729s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [..................................................X..X.XXXXX] duration: 30.027157441s FAIL (2025-09-19 20:14:37 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 3) ======LOGS====== 4D6ZGVIDMBDB74MHJKEQ7DXY5YGKQ3Q, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.050359059 (510h5m42.403144167s) [noise=true , address=145.79.227.16:4911 , piece_id=DM57ZZ7QYOCJS4MMK73CEXSQ43AGR6VVZZZ6UM4WJYDJ2RLS67DA, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.050419863 (510h5m42.403204971s) [noise=true , address=45.91.116.10:4083 , piece_id=PMWA5LTTYRMG7QKNKFOVQFBUZ7VXNGX77ZVCJWS6LFWHBYO3KW2A, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.050652169 (510h5m42.403437277s) [noise=true , address=45.91.116.6:4975 , piece_id=3C22LNVJQWLZNXHMGR77V6GEK53G4YZDPH6JPQLHOVFVFLAQ472A, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.050728963 (510h5m42.403514071s) [noise=true , address=45.91.116.9:4263 , piece_id=LMWXC45AMDOJRLIWHLWJ66TTWOMZ3YPGQKOE5UI5FD5VTWG2OJWQ, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.050878424 (510h5m42.403663521s) [noise=true , address=45.91.116.6:4973 , piece_id=NFOJBTPVGQO67SCZO3UTD3YZP5LUCFZB4NFHQ77WXASCBFZYHVMQ, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.051007326 (510h5m42.403792434s) [noise=true , address=145.79.227.12:4767 , piece_id=IRM3ZMAP5QKU7J4WSHDRKFWDHJCWVNPV6GFGEK6E5PYODDHFWBHQ, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.051095642 (510h5m42.40388074s) [noise=true , address=77.90.31.10:4517 , piece_id=PUQWUOCNTBZHVCSSVEK6SIGKZ3Z22TNX4A6KVA4TVPNTMHYPFFKQ, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.051293293 (510h5m42.404078401s) [noise=true , address=77.90.31.10:4519 , piece_id=AA75C7LRZNIZWCPVSBELM5J3Q55BOTP7J4P2763IJCTDWMA5QY2Q, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.051337946 (510h5m42.404123044s) [noise=true , address=45.91.116.4:4940 , piece_id=HELPAGD6M2TQ7G6ZMOD4TUORHA4TDXT3SEUFBXB32H26224YQPVA, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.051574239 (510h5m42.404359347s) [noise=true , address=77.90.31.11:4553 , piece_id=C7FJJWBLMSA2EBRHNKC5YPOO4H4ONZ2PB4BLU6G7RDVCQ5QECTVA, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.051578467 (510h5m42.404363575s) [noise=true , address=45.91.116.8:4045 , piece_id=T5JED3XECVVCNQSFPGPRKOLRWDMTTQIAZR2VJTFXWHT7TIECNV7A, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.051785706 (510h5m42.404570814s) [noise=true , address=77.90.31.16:4734 , piece_id=Y5Q4OS5D73UTD75M4ZSTAMJAEXR2E5GMNWYT46LNFFBEAP3KXSMQ, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.051869192 (510h5m42.40465429s) [noise=true , address=145.79.227.2:4114 , piece_id=BCMMGZR5SXZ3EQC5FN3LFAFVHO3THCJB3V4P7NEACIPDTXDWFHSQ, seg_pos={0 0}, upload=449021]: Uploading piece... Sep 19 20:14:07.115746145 (510h5m42.468531253s) [upload=449021]: Next returned nil segment Sep 19 20:14:07.115758018 (510h5m42.468543126s) [upload=449021]: Waiting for error group managing segments... Sep 19 20:14:07.124358778 (510h5m42.477143877s) [noise=true , address=145.79.227.12:4767 , piece_id=IRM3ZMAP5QKU7J4WSHDRKFWDHJCWVNPV6GFGEK6E5PYODDHFWBHQ, seg_pos={0 0}, upload=449021]: Done uploading piece. err: <nil> Sep 19 20:14:07.129956683 (510h5m42.482741781s) [noise=true , address=145.79.227.16:4911 , piece_id=DM57ZZ7QYOCJS4MMK73CEXSQ43AGR6VVZZZ6UM4WJYDJ2RLS67DA, seg_pos={0 0}, upload=449021]: Done uploading piece. err: <nil> Sep 19 20:14:07.159810388 (510h5m42.512595496s) [noise=true , address=145.79.227.2:4114 , piece_id=BCMMGZR5SXZ3EQC5FN3LFAFVHO3THCJB3V4P7NEACIPDTXDWFHSQ, seg_pos={0 0}, upload=449021]: Done uploading piece. err: <nil> Sep 19 20:14:37.013730959 (510h6m12.366516087s) [noise=true , address=45.91.116.6:4975 , piece_id=3C22LNVJQWLZNXHMGR77V6GEK53G4YZDPH6JPQLHOVFVFLAQ472A, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7, address:45.91.116.6:4975): protocol: rpc: dial tcp 45.91.116.6:4975: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.013897122 (510h6m12.366682259s) [noise=true , address=45.91.116.4:4940 , piece_id=HELPAGD6M2TQ7G6ZMOD4TUORHA4TDXT3SEUFBXB32H26224YQPVA, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.013948047 (510h6m12.366733185s) [noise=true , address=45.91.116.5:4299 , piece_id=N7YTIGUMGAIOYTWHLPABH7UBLN2PEPWZIAJRY3RBU73CA3SNLSJA, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.013996297 (510h6m12.366781426s) [noise=true , address=45.91.116.12:4226 , piece_id=NL4GWYCH2K2RSTVFF3C4L4D6ZGVIDMBDB74MHJKEQ7DXY5YGKQ3Q, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: read tcp 5.161.89.252:59380->45.91.116.12:4226: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014019571 (510h6m12.3668047s) [noise=true , address=77.90.31.10:4519 , piece_id=AA75C7LRZNIZWCPVSBELM5J3Q55BOTP7J4P2763IJCTDWMA5QY2Q, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:12rwZa7GFRR5bJTazST3crJMAuEVFwmVNLm2SnrcoZtYiHsW3FX, address:77.90.31.10:4519): protocol: read tcp 5.161.89.252:39230->77.90.31.10:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014065888 (510h6m12.366851016s) [noise=true , address=45.91.116.8:4045 , piece_id=T5JED3XECVVCNQSFPGPRKOLRWDMTTQIAZR2VJTFXWHT7TIECNV7A, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: rpc: dial tcp 45.91.116.8:4045: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014117244 (510h6m12.366902373s) [noise=true , address=77.90.31.16:4734 , piece_id=Y5Q4OS5D73UTD75M4ZSTAMJAEXR2E5GMNWYT46LNFFBEAP3KXSMQ, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: rpc: dial tcp 77.90.31.16:4734: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014166918 (510h6m12.366952045s) [noise=true , address=77.90.31.10:4517 , piece_id=PUQWUOCNTBZHVCSSVEK6SIGKZ3Z22TNX4A6KVA4TVPNTMHYPFFKQ, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: read tcp 5.161.89.252:54480->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014212653 (510h6m12.366997782s) [noise=true , address=45.91.116.9:4263 , piece_id=LMWXC45AMDOJRLIWHLWJ66TTWOMZ3YPGQKOE5UI5FD5VTWG2OJWQ, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 5.161.89.252:27582->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014231118 (510h6m12.367016246s) [noise=true , address=45.91.116.10:4083 , piece_id=PMWA5LTTYRMG7QKNKFOVQFBUZ7VXNGX77ZVCJWS6LFWHBYO3KW2A, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: rpc: dial tcp 45.91.116.10:4083: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014275300 (510h6m12.367060429s) [noise=true , address=77.90.31.11:4553 , piece_id=C7FJJWBLMSA2EBRHNKC5YPOO4H4ONZ2PB4BLU6G7RDVCQ5QECTVA, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:20170->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014320045 (510h6m12.367105172s) [noise=true , address=77.90.31.19:4841 , piece_id=ROB4BOWZTDJR5AZAG5IEFM6QGEEKR4YX6E4TKDLLHR47SPFQB7BQ, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:42656->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014351574 (510h6m12.367136712s) [noise=true , address=45.91.116.6:4973 , piece_id=NFOJBTPVGQO67SCZO3UTD3YZP5LUCFZB4NFHQ77WXASCBFZYHVMQ, seg_pos={0 0}, upload=449021]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:25536->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:14:37.014380297 (510h6m12.367165426s) [seg_pos={0 0}, upload=449021]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil> Sep 19 20:14:37.038950226 (510h6m12.391735354s) [upload=449021]: Done uploading segments... Sep 19 20:14:37.038971025 (510h6m12.391756153s) [upload=449021]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3) Duration: 30.027176346s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XX.XX] duration: 30.000667842s FAIL (2025-09-19 20:14:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000686157s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................................................X..X.XXXX] duration: 30.017462896s FAIL (2025-09-19 20:13:39 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== [seg_pos={0 0}, upload=449007]: Begin upload segment... Sep 19 20:13:09.242858348 (510h4m44.595643475s) [seg_pos={0 0}, upload=449007]: Done begin upload segment. Sep 19 20:13:09.242888204 (510h4m44.595673331s) [noise=true , address=145.79.227.4:4186 , piece_id=WDORHGCNFNBQHHPB2QBEZP26XZIQ7SED3Y2J4AN66LVXW3D3G57Q, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.242970768 (510h4m44.595755867s) [noise=true , address=45.91.116.13:4155 , piece_id=UZFNALFKVJBKR463UGZZL2ITKHF5Y5ZLHET3YYI6U266SMTB242A, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.243340884 (510h4m44.596125992s) [noise=true , address=77.90.31.16:4734 , piece_id=D56MUL2KEVKS6GMMXSHTZRSMXK45HRMKKVHC7I6WTGHMHYBKL6OA, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.243378555 (510h4m44.596163682s) [noise=true , address=45.91.116.6:4974 , piece_id=BJPKLM4GUSB5I24CFMH5QLPEVKSW6MXZRIEFEKEUAYUNWBZUVY7A, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.243625357 (510h4m44.596410455s) [noise=true , address=145.79.227.17:4970 , piece_id=WZGKXHGEOBTA3VVA4ASMWJKORNXQR2L2H46VA34FLUUZNVQD2YCA, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.243777593 (510h4m44.596562721s) [noise=true , address=145.79.227.14:4840 , piece_id=MM2HGRFMH4NMSZB3FMGZDZY4DQPBG326HADLXPCYJIEZQDQKM3RA, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.243859497 (510h4m44.596644594s) [noise=true , address=145.79.227.7:4294 , piece_id=REEQ5R3FMEOBNYJRHU72UNMHFHOMXW7O67JGBJ3SWXPIK3SSK7DQ, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.244122531 (510h4m44.596907639s) [noise=true , address=145.79.227.15:4876 , piece_id=73QXTRCKFDCVI4RAVHERTZ3E7YDCHMSBRD7IOSKDTXJJQGWZEY4A, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.244237777 (510h4m44.597022915s) [noise=true , address=45.91.116.4:4939 , piece_id=B2S5BVGCO3CSFPVKENJ63RSW5L44A6GER3GPFTGLZWC4HNWZCQJA, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.244351329 (510h4m44.597136438s) [noise=true , address=77.90.31.18:4805 , piece_id=ZKZBRHUEFL6PN4S6ZJ3NVKKBE7I7QWUO35UD4BFU6EEBUGGUVCLQ, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.244626095 (510h4m44.597411203s) [noise=true , address=77.90.31.21:4879 , piece_id=DZGQC4CUGNQ2KIYGUA3CLQUNLKP7JDFMOVAVUWVMMRQD5RF23KUA, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.244651683 (510h4m44.597436821s) [noise=true , address=45.91.116.10:4083 , piece_id=DQ7VFW5S5FNWFCKQ42RNW4WSCCDZKWUIXC3UZFILR6XSK4VFY2IQ, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.245020224 (510h4m44.597805362s) [noise=true , address=145.79.227.4:4187 , piece_id=MQXD5Q3WZXOFDGZJF5CJRGGL3STNHM4E2GQ2QHZA2C4A2PTACNQQ, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.245374460 (510h4m44.598159588s) [noise=true , address=77.90.31.13:4628 , piece_id=GGCRRIKZAUJSZD4RA2R6AHTNAQO7KRHKMIVUCICQV4V5RULTNJHQ, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.245794107 (510h4m44.598579235s) [noise=true , address=45.91.116.9:4263 , piece_id=2VUWKHEHNMW33LPHUX2SWG56P7QWE3HRZIKMC3BY6NLDCDQAOALA, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.246201842 (510h4m44.598986971s) [noise=true , address=77.90.31.12:4590 , piece_id=5EXTM2ERF33GRVF7LZ2JSNAC3HT6ZLUROBKHC6XQME23KCXUBKFA, seg_pos={0 0}, upload=449007]: Uploading piece... Sep 19 20:13:09.315134893 (510h4m44.667920001s) [upload=449007]: Next returned nil segment Sep 19 20:13:09.315146084 (510h4m44.667931182s) [upload=449007]: Waiting for error group managing segments... Sep 19 20:13:09.323613485 (510h4m44.676398582s) [noise=true , address=145.79.227.7:4294 , piece_id=REEQ5R3FMEOBNYJRHU72UNMHFHOMXW7O67JGBJ3SWXPIK3SSK7DQ, seg_pos={0 0}, upload=449007]: Done uploading piece. err: <nil> Sep 19 20:13:09.324558277 (510h4m44.677343386s) [noise=true , address=145.79.227.15:4876 , piece_id=73QXTRCKFDCVI4RAVHERTZ3E7YDCHMSBRD7IOSKDTXJJQGWZEY4A, seg_pos={0 0}, upload=449007]: Done uploading piece. err: <nil> Sep 19 20:13:09.324605547 (510h4m44.677390674s) [noise=true , address=145.79.227.4:4186 , piece_id=WDORHGCNFNBQHHPB2QBEZP26XZIQ7SED3Y2J4AN66LVXW3D3G57Q, seg_pos={0 0}, upload=449007]: Done uploading piece. err: <nil> Sep 19 20:13:09.325325908 (510h4m44.678111007s) [noise=true , address=145.79.227.17:4970 , piece_id=WZGKXHGEOBTA3VVA4ASMWJKORNXQR2L2H46VA34FLUUZNVQD2YCA, seg_pos={0 0}, upload=449007]: Done uploading piece. err: <nil> Sep 19 20:13:09.327345097 (510h4m44.680130225s) [noise=true , address=145.79.227.14:4840 , piece_id=MM2HGRFMH4NMSZB3FMGZDZY4DQPBG326HADLXPCYJIEZQDQKM3RA, seg_pos={0 0}, upload=449007]: Done uploading piece. err: <nil> Sep 19 20:13:09.331963995 (510h4m44.684749123s) [noise=true , address=145.79.227.4:4187 , piece_id=MQXD5Q3WZXOFDGZJF5CJRGGL3STNHM4E2GQ2QHZA2C4A2PTACNQQ, seg_pos={0 0}, upload=449007]: Done uploading piece. err: <nil> Sep 19 20:13:39.156430981 (510h5m14.509216109s) [noise=true , address=45.91.116.13:4155 , piece_id=UZFNALFKVJBKR463UGZZL2ITKHF5Y5ZLHET3YYI6U266SMTB242A, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: rpc: dial tcp 45.91.116.13:4155: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.156458773 (510h5m14.509243902s) [noise=true , address=77.90.31.21:4879 , piece_id=DZGQC4CUGNQ2KIYGUA3CLQUNLKP7JDFMOVAVUWVMMRQD5RF23KUA, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.156799131 (510h5m14.50958426s) [noise=true , address=77.90.31.18:4805 , piece_id=ZKZBRHUEFL6PN4S6ZJ3NVKKBE7I7QWUO35UD4BFU6EEBUGGUVCLQ, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT, address:77.90.31.18:4805): protocol: read tcp 5.161.89.252:16172->77.90.31.18:4805: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.156863693 (510h5m14.509648821s) [noise=true , address=77.90.31.12:4590 , piece_id=5EXTM2ERF33GRVF7LZ2JSNAC3HT6ZLUROBKHC6XQME23KCXUBKFA, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: rpc: dial tcp 77.90.31.12:4590: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.156907305 (510h5m14.509692433s) [noise=true , address=77.90.31.13:4628 , piece_id=GGCRRIKZAUJSZD4RA2R6AHTNAQO7KRHKMIVUCICQV4V5RULTNJHQ, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: read tcp 5.161.89.252:31736->77.90.31.13:4628: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.156941699 (510h5m14.509726827s) [noise=true , address=45.91.116.6:4974 , piece_id=BJPKLM4GUSB5I24CFMH5QLPEVKSW6MXZRIEFEKEUAYUNWBZUVY7A, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.156959703 (510h5m14.509744841s) [noise=true , address=77.90.31.16:4734 , piece_id=D56MUL2KEVKS6GMMXSHTZRSMXK45HRMKKVHC7I6WTGHMHYBKL6OA, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, address:77.90.31.16:4734): protocol: read tcp 5.161.89.252:43364->77.90.31.16:4734: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.156996081 (510h5m14.509781209s) [noise=true , address=45.91.116.10:4083 , piece_id=DQ7VFW5S5FNWFCKQ42RNW4WSCCDZKWUIXC3UZFILR6XSK4VFY2IQ, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 5.161.89.252:60664->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.157010539 (510h5m14.509795666s) [noise=true , address=45.91.116.4:4939 , piece_id=B2S5BVGCO3CSFPVKENJ63RSW5L44A6GER3GPFTGLZWC4HNWZCQJA, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: read tcp 5.161.89.252:46520->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.157065832 (510h5m14.50985097s) [noise=true , address=45.91.116.9:4263 , piece_id=2VUWKHEHNMW33LPHUX2SWG56P7QWE3HRZIKMC3BY6NLDCDQAOALA, seg_pos={0 0}, upload=449007]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 5.161.89.252:26460->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:13:39.157121196 (510h5m14.509906324s) [seg_pos={0 0}, upload=449007]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Sep 19 20:13:39.171767782 (510h5m14.524552911s) [upload=449007]: Done uploading segments... Sep 19 20:13:39.171786257 (510h5m14.524571385s) [upload=449007]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.017482413s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................XX.X] duration: 30.000835643s FAIL (2025-09-19 20:13:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000858846s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [....................................................X..X.XXX] duration: 30.017084784s FAIL (2025-09-19 20:12:47 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== ad=448991]: Uploading piece... Sep 19 20:12:17.541980529 (510h3m52.894765627s) [noise=true , address=77.90.31.15:4699 , piece_id=3ZVLTLUSRFH4BKXRUUNPIHBRS653HZYN2ILVT36EY3MZ6UNK6UXQ, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.541998242 (510h3m52.89478334s) [noise=true , address=77.90.31.13:4627 , piece_id=4L25YRXZOD72CRQWEOSEGR22HGLAUOTM4DY6FIW2PB3TW3YISVBA, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.542227101 (510h3m52.895012209s) [noise=true , address=45.91.116.8:4047 , piece_id=VA65IDX3RXIJCADSJF5KOT7QSMGO47V4UCBX3VZAHHOHDKYCYDTA, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.542263610 (510h3m52.895048718s) [noise=true , address=77.90.31.21:4879 , piece_id=AGZVZIH3SP5OH2Z44UMNGOURRVVSXWMR67F35DVYI6JW7H43GKSA, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.542505875 (510h3m52.895290973s) [noise=true , address=45.91.116.7:4010 , piece_id=HXIS3DCIJKSX4VK65J7NFPQVZC3TB3RIHXOKHI3WIYJB4P7TF4BA, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.542541292 (510h3m52.895326389s) [noise=true , address=45.91.116.12:4225 , piece_id=TW2WU4S57BKQEJ37B2P7SEPAOZ4MO7BSEMQ7Z4LQWDKEBYXPUCBQ, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.542811578 (510h3m52.895596667s) [noise=true , address=145.79.227.3:4150 , piece_id=3JTRPP44ZB4DILCI7OYBZYTALJ7CY4RDHRMCID6P6UTXELJKOH2Q, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.542820074 (510h3m52.895605182s) [noise=true , address=145.79.227.7:4294 , piece_id=DF65JATXZVDP6PESWKKZZTFTHNGZCTR2ZV7CKKF7AZNOPEB6QJGQ, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.543008368 (510h3m52.895793465s) [noise=true , address=145.79.227.4:4187 , piece_id=XMAMVOXGQPKTAR5RQ6X5KFWFETV2YPQYPJV2LDMASYI5F54HADFQ, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.543113445 (510h3m52.895898553s) [noise=true , address=45.91.116.11:4119 , piece_id=DUQAUZZLRKKDVUNQXAORQH7NRKRNDRCE7L2SXK2JBJHISXLYG3MQ, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.543271581 (510h3m52.896056679s) [noise=true , address=77.90.31.12:4592 , piece_id=MVINYQHHU6MVF3GO3RDRMURNLC4AZHGW6XPFLW7D4ERY6IYDOYUA, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.543541829 (510h3m52.896326927s) [noise=true , address=77.90.31.17:4772 , piece_id=CNODJIKZRCI4WPFG47VRE5J4RNP5JCEJPFUIDB4K7NRRYHTW2H3A, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.543735672 (510h3m52.89652077s) [noise=true , address=145.79.227.18:4007 , piece_id=PVBLTOQOTEHOZSFYTOMOTORJTJUMM6Q6FSBTGNXLUBYA4HOOIUIQ, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.546518304 (510h3m52.899303402s) [noise=true , address=45.91.116.7:4011 , piece_id=DTIV5UOLYTRRNKQ3TIGL6JKCGOV7NQ4FVRIW4VLWEKMRNHSYZYIQ, seg_pos={0 0}, upload=448991]: Uploading piece... Sep 19 20:12:17.610994392 (510h3m52.9637795s) [upload=448991]: Next returned nil segment Sep 19 20:12:17.611006825 (510h3m52.963791933s) [upload=448991]: Waiting for error group managing segments... Sep 19 20:12:17.621487422 (510h3m52.97427253s) [noise=true , address=145.79.227.18:4007 , piece_id=PVBLTOQOTEHOZSFYTOMOTORJTJUMM6Q6FSBTGNXLUBYA4HOOIUIQ, seg_pos={0 0}, upload=448991]: Done uploading piece. err: <nil> Sep 19 20:12:17.622856332 (510h3m52.97564143s) [noise=true , address=145.79.227.4:4187 , piece_id=XMAMVOXGQPKTAR5RQ6X5KFWFETV2YPQYPJV2LDMASYI5F54HADFQ, seg_pos={0 0}, upload=448991]: Done uploading piece. err: <nil> Sep 19 20:12:17.650309471 (510h3m53.003094579s) [noise=true , address=145.79.227.7:4294 , piece_id=DF65JATXZVDP6PESWKKZZTFTHNGZCTR2ZV7CKKF7AZNOPEB6QJGQ, seg_pos={0 0}, upload=448991]: Done uploading piece. err: <nil> Sep 19 20:12:17.656222016 (510h3m53.009007114s) [noise=true , address=145.79.227.3:4150 , piece_id=3JTRPP44ZB4DILCI7OYBZYTALJ7CY4RDHRMCID6P6UTXELJKOH2Q, seg_pos={0 0}, upload=448991]: Done uploading piece. err: <nil> Sep 19 20:12:47.508083068 (510h4m22.860868167s) [noise=true , address=45.91.116.12:4225 , piece_id=TW2WU4S57BKQEJ37B2P7SEPAOZ4MO7BSEMQ7Z4LQWDKEBYXPUCBQ, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: rpc: dial tcp 45.91.116.12:4225: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508126239 (510h4m22.860911328s) [noise=true , address=77.90.31.17:4772 , piece_id=CNODJIKZRCI4WPFG47VRE5J4RNP5JCEJPFUIDB4K7NRRYHTW2H3A, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: rpc: dial tcp 77.90.31.17:4772: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508137912 (510h4m22.860923009s) [noise=true , address=45.91.116.7:4010 , piece_id=HXIS3DCIJKSX4VK65J7NFPQVZC3TB3RIHXOKHI3WIYJB4P7TF4BA, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: read tcp 5.161.89.252:45862->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508154492 (510h4m22.86093959s) [noise=true , address=77.90.31.12:4592 , piece_id=MVINYQHHU6MVF3GO3RDRMURNLC4AZHGW6XPFLW7D4ERY6IYDOYUA, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508177014 (510h4m22.860962113s) [noise=true , address=45.91.116.8:4047 , piece_id=VA65IDX3RXIJCADSJF5KOT7QSMGO47V4UCBX3VZAHHOHDKYCYDTA, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN, address:45.91.116.8:4047): protocol: read tcp 5.161.89.252:56004->45.91.116.8:4047: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508211129 (510h4m22.860996237s) [noise=true , address=45.91.116.7:4011 , piece_id=DTIV5UOLYTRRNKQ3TIGL6JKCGOV7NQ4FVRIW4VLWEKMRNHSYZYIQ, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:12gZ5sVLg58D2gh3MVkHJ42qVRWrRU5EbuDWcG6hYU9MPgcSNQf, address:45.91.116.7:4011): protocol: read tcp 5.161.89.252:33076->45.91.116.7:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508242177 (510h4m22.861027285s) [noise=true , address=45.91.116.14:4189 , piece_id=XCGQ42ZMOZ7EJWICT6RADIE64IDX4JJ47TESQJEV5GRJ463HXWRA, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:27656->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508262084 (510h4m22.861047192s) [noise=true , address=77.90.31.13:4627 , piece_id=4L25YRXZOD72CRQWEOSEGR22HGLAUOTM4DY6FIW2PB3TW3YISVBA, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB, address:77.90.31.13:4627): protocol: read tcp 5.161.89.252:18628->77.90.31.13:4627: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508309193 (510h4m22.86109429s) [noise=true , address=77.90.31.21:4879 , piece_id=AGZVZIH3SP5OH2Z44UMNGOURRVVSXWMR67F35DVYI6JW7H43GKSA, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: read tcp 5.161.89.252:28030->77.90.31.21:4879: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508334811 (510h4m22.861119909s) [noise=true , address=45.91.116.11:4119 , piece_id=DUQAUZZLRKKDVUNQXAORQH7NRKRNDRCE7L2SXK2JBJHISXLYG3MQ, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm, address:45.91.116.11:4119): protocol: read tcp 5.161.89.252:54484->45.91.116.11:4119: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508355129 (510h4m22.861140237s) [noise=true , address=77.90.31.15:4699 , piece_id=3ZVLTLUSRFH4BKXRUUNPIHBRS653HZYN2ILVT36EY3MZ6UNK6UXQ, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: read tcp 5.161.89.252:17554->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508360038 (510h4m22.861145136s) [noise=true , address=77.90.31.10:4518 , piece_id=3JWX6LPNXGUDO47GW2GPA45SX5G4GE6HHMYLSVCM55HL67VPHJFA, seg_pos={0 0}, upload=448991]: Done uploading piece. err: ecclient: upload failed (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, address:77.90.31.10:4518): protocol: read tcp 5.161.89.252:58460->77.90.31.10:4518: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:12:47.508375938 (510h4m22.861161036s) [seg_pos={0 0}, upload=448991]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Sep 19 20:12:47.523411074 (510h4m22.876196172s) [upload=448991]: Done uploading segments... Sep 19 20:12:47.523424189 (510h4m22.876209297s) [upload=448991]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.017101556s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................X..X.XX] duration: 30.014747268s FAIL (2025-09-19 20:11:50 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== 910406572s) [upload=448977]: Done starting upload Sep 19 20:11:20.557679753 (510h2m55.910464861s) [upload=448977]: Uploading segments... Sep 19 20:11:20.557832420 (510h2m55.910617528s) [upload=448977]: Got next segment. Inline: false Sep 19 20:11:20.557857486 (510h2m55.910642594s) [upload=448977]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 19 20:11:20.557860943 (510h2m55.910646051s) [upload=448977]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 19 20:11:20.580231273 (510h2m55.933016382s) [seg_pos={0 0}, upload=448977]: Begin upload segment... Sep 19 20:11:20.580262141 (510h2m55.933047239s) [seg_pos={0 0}, upload=448977]: Done begin upload segment. Sep 19 20:11:20.580281578 (510h2m55.933066686s) [noise=true , address=45.91.116.9:4262 , piece_id=WNRZZKDMDTZPPYO4JVMJCXZ3MTCNYK7YFUDRXFJAOWMUDYL3UTIQ, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.580306796 (510h2m55.933091904s) [noise=true , address=145.79.227.2:4114 , piece_id=XKG74IK4QOH7O5AWPIJ5KEZCET3XVCGPLB4VCOTORRP75FJPFCZQ, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.580573756 (510h2m55.933358854s) [noise=true , address=145.79.227.19:4077 , piece_id=UWBDUN64XY2PW2MMZQH2PZM3RTZESB6EG2X5TE2NYXXZFAOCUJSA, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.580590317 (510h2m55.933375415s) [noise=true , address=145.79.227.14:4837 , piece_id=JT5XGKYUBDVBDBNUPU73FE7ECQTFTK62KRIIOZL55ZMSTLAMG2XQ, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.580790163 (510h2m55.933575271s) [noise=true , address=145.79.227.18:4005 , piece_id=2GRQXNHUVJ2WAKNB6EGJHQZJCS3WODDRB75NHKGSYBZYD3MSD75A, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.580847350 (510h2m55.933632448s) [noise=true , address=45.91.116.8:4048 , piece_id=MI3RPBPPZLRSKW6VIWD7W6RT64SMAHDYKWOZALSNNPUVPGKC3RKQ, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.580999475 (510h2m55.933784583s) [noise=true , address=145.79.227.16:4910 , piece_id=4GLZWE2LC4M66PT3FRZR7RL6F3J6R32FTK5GQMB3DCSTJJIQP2NA, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.581076640 (510h2m55.933861738s) [noise=true , address=45.91.116.7:4010 , piece_id=4CNYFURWGL3TJ5QVJNKLYSSLZJDO2EDEFWKMUAZPYUTFJQEHTAVQ, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.581217474 (510h2m55.934002573s) [noise=true , address=145.79.227.17:4969 , piece_id=TWPQCWW44YRU6YMIITZYXEAKBBAVU3PLLRADVHHPV3XN4EPMHUJQ, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.581293146 (510h2m55.934078254s) [noise=true , address=45.91.116.13:4155 , piece_id=CH4OTXGQAVUEZLFHOBUOEL65BDRP2DFXEJKBAD24WWBPYQDG4Y2Q, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.581434592 (510h2m55.934219689s) [noise=true , address=145.79.227.12:4767 , piece_id=THIBYVXZCSGBQ5YRW6IV7PLVFMID6WEMCTUYNGEY5HF43BBBH4OA, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.581516815 (510h2m55.934301914s) [noise=true , address=77.90.31.19:4843 , piece_id=CN3CGIU4PTDIWWAAMRETXIVR25BRA3TKLUGC2ETRDHIBCVGXJOFQ, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.581656289 (510h2m55.934441397s) [noise=true , address=77.90.31.10:4520 , piece_id=QSGMDAHXIXZQ7UZICFKGXU7JBXCGNM7KDCZ6QAEOTVDV3Y3KDVNQ, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.581754393 (510h2m55.934539501s) [noise=true , address=145.79.227.7:4293 , piece_id=EZOCVGQZQEQLSQGWNFMR7ROCS63SFNBUWLP4ETCE3N4HO32WIJ7Q, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.581868506 (510h2m55.934653604s) [noise=true , address=45.91.116.5:4299 , piece_id=7R5PNK36PXKYR3GW5CVJIHWOZDVXUKPJEN2ZXMKLOFY7AUPQIZIA, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.582068050 (510h2m55.934853158s) [noise=true , address=45.91.116.12:4226 , piece_id=B3WXVOVZD35XV4B7H3K5BSC22VEACF6BVSBAA6VB4TXLTUTJ2TQQ, seg_pos={0 0}, upload=448977]: Uploading piece... Sep 19 20:11:20.672601959 (510h2m56.025387087s) [upload=448977]: Next returned nil segment Sep 19 20:11:20.672617708 (510h2m56.025402836s) [upload=448977]: Waiting for error group managing segments... Sep 19 20:11:20.680307028 (510h2m56.033092155s) [noise=true , address=145.79.227.17:4969 , piece_id=TWPQCWW44YRU6YMIITZYXEAKBBAVU3PLLRADVHHPV3XN4EPMHUJQ, seg_pos={0 0}, upload=448977]: Done uploading piece. err: <nil> Sep 19 20:11:20.680435449 (510h2m56.033220568s) [noise=true , address=145.79.227.14:4837 , piece_id=JT5XGKYUBDVBDBNUPU73FE7ECQTFTK62KRIIOZL55ZMSTLAMG2XQ, seg_pos={0 0}, upload=448977]: Done uploading piece. err: <nil> Sep 19 20:11:20.681567533 (510h2m56.034352662s) [noise=true , address=145.79.227.19:4077 , piece_id=UWBDUN64XY2PW2MMZQH2PZM3RTZESB6EG2X5TE2NYXXZFAOCUJSA, seg_pos={0 0}, upload=448977]: Done uploading piece. err: <nil> Sep 19 20:11:20.684159087 (510h2m56.036944215s) [noise=true , address=145.79.227.2:4114 , piece_id=XKG74IK4QOH7O5AWPIJ5KEZCET3XVCGPLB4VCOTORRP75FJPFCZQ, seg_pos={0 0}, upload=448977]: Done uploading piece. err: <nil> Sep 19 20:11:20.685308414 (510h2m56.038093532s) [noise=true , address=145.79.227.12:4767 , piece_id=THIBYVXZCSGBQ5YRW6IV7PLVFMID6WEMCTUYNGEY5HF43BBBH4OA, seg_pos={0 0}, upload=448977]: Done uploading piece. err: <nil> Sep 19 20:11:20.686622641 (510h2m56.039407779s) [noise=true , address=145.79.227.16:4910 , piece_id=4GLZWE2LC4M66PT3FRZR7RL6F3J6R32FTK5GQMB3DCSTJJIQP2NA, seg_pos={0 0}, upload=448977]: Done uploading piece. err: <nil> Sep 19 20:11:20.689439607 (510h2m56.042224765s) [noise=true , address=145.79.227.7:4293 , piece_id=EZOCVGQZQEQLSQGWNFMR7ROCS63SFNBUWLP4ETCE3N4HO32WIJ7Q, seg_pos={0 0}, upload=448977]: Done uploading piece. err: <nil> Sep 19 20:11:20.759228373 (510h2m56.112013501s) [noise=true , address=145.79.227.18:4005 , piece_id=2GRQXNHUVJ2WAKNB6EGJHQZJCS3WODDRB75NHKGSYBZYD3MSD75A, seg_pos={0 0}, upload=448977]: Done uploading piece. err: <nil> Sep 19 20:11:50.537147856 (510h3m25.889932965s) [noise=true , address=77.90.31.10:4520 , piece_id=QSGMDAHXIXZQ7UZICFKGXU7JBXCGNM7KDCZ6QAEOTVDV3Y3KDVNQ, seg_pos={0 0}, upload=448977]: Done uploading piece. err: ecclient: upload failed (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp, address:77.90.31.10:4520): protocol: read tcp 5.161.89.252:22284->77.90.31.10:4520: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:50.537237394 (510h3m25.890022492s) [noise=true , address=45.91.116.12:4226 , piece_id=B3WXVOVZD35XV4B7H3K5BSC22VEACF6BVSBAA6VB4TXLTUTJ2TQQ, seg_pos={0 0}, upload=448977]: Done uploading piece. err: ecclient: upload failed (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1, address:45.91.116.12:4226): protocol: rpc: dial tcp 45.91.116.12:4226: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:50.537399769 (510h3m25.890184887s) [noise=true , address=45.91.116.9:4262 , piece_id=WNRZZKDMDTZPPYO4JVMJCXZ3MTCNYK7YFUDRXFJAOWMUDYL3UTIQ, seg_pos={0 0}, upload=448977]: Done uploading piece. err: ecclient: upload failed (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto, address:45.91.116.9:4262): protocol: read tcp 5.161.89.252:30292->45.91.116.9:4262: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:50.537426299 (510h3m25.890211407s) [noise=true , address=45.91.116.5:4299 , piece_id=7R5PNK36PXKYR3GW5CVJIHWOZDVXUKPJEN2ZXMKLOFY7AUPQIZIA, seg_pos={0 0}, upload=448977]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: read tcp 5.161.89.252:39402->45.91.116.5:4299: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:50.537490519 (510h3m25.890275627s) [noise=true , address=45.91.116.13:4155 , piece_id=CH4OTXGQAVUEZLFHOBUOEL65BDRP2DFXEJKBAD24WWBPYQDG4Y2Q, seg_pos={0 0}, upload=448977]: Done uploading piece. err: ecclient: upload failed (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf, address:45.91.116.13:4155): protocol: rpc: dial tcp 45.91.116.13:4155: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:50.537537296 (510h3m25.890322395s) [noise=true , address=45.91.116.8:4048 , piece_id=MI3RPBPPZLRSKW6VIWD7W6RT64SMAHDYKWOZALSNNPUVPGKC3RKQ, seg_pos={0 0}, upload=448977]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: rpc: dial tcp 45.91.116.8:4048: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:50.537571681 (510h3m25.890356789s) [noise=true , address=45.91.116.7:4010 , piece_id=4CNYFURWGL3TJ5QVJNKLYSSLZJDO2EDEFWKMUAZPYUTFJQEHTAVQ, seg_pos={0 0}, upload=448977]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: read tcp 5.161.89.252:44412->45.91.116.7:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:50.537648004 (510h3m25.890433103s) [noise=true , address=77.90.31.19:4843 , piece_id=CN3CGIU4PTDIWWAAMRETXIVR25BRA3TKLUGC2ETRDHIBCVGXJOFQ, seg_pos={0 0}, upload=448977]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: read tcp 5.161.89.252:21858->77.90.31.19:4843: use of closed network connection; context deadline exceeded Sep 19 20:11:50.537685916 (510h3m25.890471024s) [seg_pos={0 0}, upload=448977]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Sep 19 20:11:50.550912988 (510h3m25.903698106s) [upload=448977]: Done uploading segments... Sep 19 20:11:50.550927285 (510h3m25.903712403s) [upload=448977]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 30.01475921s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 505.480516ms FAIL (2025-09-19 20:11:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 505.502446ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.00103635s FAIL (2025-09-19 20:11:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.001059373s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................X..X.X] duration: 30.01764089s FAIL (2025-09-19 20:11:20 UTC) operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 0:10:50.470063045 (510h2m25.822848153s) [noise=true , address=145.79.227.11:4729 , piece_id=H7GISXLFTAUCTKXZVCVW6HPGBOS57JTPKSUY7C54YSSJQRYFWUOA, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.470216903 (510h2m25.823002031s) [noise=true , address=45.91.116.5:4297 , piece_id=BCSTXIIO2MSEGGVGOJN3DQ26LF5M42TKOPX5RSU7GLVKJUXVBNWQ, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.470304057 (510h2m25.823089165s) [noise=true , address=45.91.116.14:4191 , piece_id=HVH357AAWHCBEOUAKK72WNKCFK7QMHWDGGDQ4RTM3F4VKPP7NKDQ, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.470540430 (510h2m25.823325538s) [noise=true , address=77.90.31.21:4878 , piece_id=U3MMMM5VIZUS7M2EBLUBNSIOEYKXUSCFCLQGTOKT4E3MJVFFTYGA, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.470608167 (510h2m25.823393295s) [noise=true , address=77.90.31.14:4663 , piece_id=G2MXIN743O7SEVX2A42KZN3KUCCYRVGFHDRSGBOJDQAOTG3AOCZQ, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.470756666 (510h2m25.823541774s) [noise=true , address=45.91.116.11:4117 , piece_id=I7LL33KZLOIHOHXINO4XRB7JLWETGEVRKB6USMWC3X7MS6IT547Q, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.470969556 (510h2m25.823754664s) [noise=true , address=45.91.116.8:4045 , piece_id=XEOBGYQ7GUSQNURJJSVFXM232TNME23RD6KW7ZLKLUGDGXCTVJSQ, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.470987769 (510h2m25.823772908s) [noise=true , address=45.91.116.9:4263 , piece_id=6CT342LXL4RCRDAEFPOBTWQFFGZZTIXRZ6ZW3HS67L3XAA5OEYUA, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.471261122 (510h2m25.82404623s) [noise=true , address=77.90.31.16:4735 , piece_id=S6LGANW2YTRHNO7MDCG76GDWHM7SQRVPMV542KI6BI4MV2EVEB4A, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.471492186 (510h2m25.824277284s) [noise=true , address=77.90.31.17:4772 , piece_id=SG22ECXYCYMZWI5RGZAQCMMJWO4WGKWCD4CF5HXMP56ZDN4XNG5Q, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.471577355 (510h2m25.824362484s) [noise=true , address=145.79.227.13:4802 , piece_id=MRB6QTG46VDLWE3RZVU5OPLV7MNX3234RMHNFASULW4QTJAHEPWA, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.471752355 (510h2m25.824537453s) [noise=true , address=45.91.116.13:4154 , piece_id=RRLXBP36ZKJUBIYPN5TNOZECNE5WLHEGUXNMYQN6L4DPKA6A34BA, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.471997675 (510h2m25.824782803s) [noise=true , address=77.90.31.15:4699 , piece_id=XQMZA56M7WY5GWQDGH7N22UFLH335B235VORWRLH2YEIP2QBO7QA, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.472421580 (510h2m25.825206708s) [noise=true , address=145.79.227.3:4149 , piece_id=4CTIQUFLWIERJOOAITMLT2IJCUNKTYQHQDXHJ2EPKWG6OZTD4WYA, seg_pos={0 0}, upload=448970]: Uploading piece... Sep 19 20:10:50.543012042 (510h2m25.89579717s) [upload=448970]: Next returned nil segment Sep 19 20:10:50.543028843 (510h2m25.895813981s) [upload=448970]: Waiting for error group managing segments... Sep 19 20:10:50.552241091 (510h2m25.905026199s) [noise=true , address=145.79.227.13:4802 , piece_id=MRB6QTG46VDLWE3RZVU5OPLV7MNX3234RMHNFASULW4QTJAHEPWA, seg_pos={0 0}, upload=448970]: Done uploading piece. err: <nil> Sep 19 20:10:50.553505264 (510h2m25.906290402s) [noise=true , address=145.79.227.4:4186 , piece_id=ADNI7MQC5SAZR45IPSM4LD2ECP63ZUTUPSOY6HWEAAPPCXV6LX4A, seg_pos={0 0}, upload=448970]: Done uploading piece. err: <nil> Sep 19 20:10:50.553678258 (510h2m25.906463357s) [noise=true , address=145.79.227.3:4149 , piece_id=4CTIQUFLWIERJOOAITMLT2IJCUNKTYQHQDXHJ2EPKWG6OZTD4WYA, seg_pos={0 0}, upload=448970]: Done uploading piece. err: <nil> Sep 19 20:10:50.554574911 (510h2m25.907360039s) [noise=true , address=145.79.227.11:4729 , piece_id=H7GISXLFTAUCTKXZVCVW6HPGBOS57JTPKSUY7C54YSSJQRYFWUOA, seg_pos={0 0}, upload=448970]: Done uploading piece. err: <nil> Sep 19 20:11:20.432550804 (510h2m55.785335912s) [noise=true , address=77.90.31.21:4878 , piece_id=U3MMMM5VIZUS7M2EBLUBNSIOEYKXUSCFCLQGTOKT4E3MJVFFTYGA, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK, address:77.90.31.21:4878): protocol: rpc: dial tcp 77.90.31.21:4878: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432614193 (510h2m55.78539929s) [noise=true , address=45.91.116.14:4191 , piece_id=HVH357AAWHCBEOUAKK72WNKCFK7QMHWDGGDQ4RTM3F4VKPP7NKDQ, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432762250 (510h2m55.785547358s) [noise=true , address=77.90.31.14:4663 , piece_id=G2MXIN743O7SEVX2A42KZN3KUCCYRVGFHDRSGBOJDQAOTG3AOCZQ, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE, address:77.90.31.14:4663): protocol: read tcp 5.161.89.252:13638->77.90.31.14:4663: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432839084 (510h2m55.785624192s) [noise=true , address=77.90.31.16:4735 , piece_id=S6LGANW2YTRHNO7MDCG76GDWHM7SQRVPMV542KI6BI4MV2EVEB4A, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: read tcp 5.161.89.252:56510->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432854524 (510h2m55.785639632s) [noise=true , address=45.91.116.13:4154 , piece_id=RRLXBP36ZKJUBIYPN5TNOZECNE5WLHEGUXNMYQN6L4DPKA6A34BA, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: read tcp 5.161.89.252:38108->45.91.116.13:4154: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432871745 (510h2m55.785656853s) [noise=true , address=45.91.116.11:4117 , piece_id=I7LL33KZLOIHOHXINO4XRB7JLWETGEVRKB6USMWC3X7MS6IT547Q, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 5.161.89.252:30826->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432886944 (510h2m55.785672052s) [noise=true , address=77.90.31.17:4772 , piece_id=SG22ECXYCYMZWI5RGZAQCMMJWO4WGKWCD4CF5HXMP56ZDN4XNG5Q, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 5.161.89.252:55642->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432900971 (510h2m55.785686079s) [noise=true , address=45.91.116.9:4263 , piece_id=6CT342LXL4RCRDAEFPOBTWQFFGZZTIXRZ6ZW3HS67L3XAA5OEYUA, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, address:45.91.116.9:4263): protocol: read tcp 5.161.89.252:23000->45.91.116.9:4263: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432918333 (510h2m55.785703441s) [noise=true , address=77.90.31.11:4553 , piece_id=2KNW35VC4I7M73DOCEDRW37JRI2ZCMYKMZ7PAJOJZF4FYBQWNVBA, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:15564->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432931217 (510h2m55.785716315s) [noise=true , address=77.90.31.15:4699 , piece_id=XQMZA56M7WY5GWQDGH7N22UFLH335B235VORWRLH2YEIP2QBO7QA, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: read tcp 5.161.89.252:15558->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432945534 (510h2m55.785730642s) [noise=true , address=45.91.116.8:4045 , piece_id=XEOBGYQ7GUSQNURJJSVFXM232TNME23RD6KW7ZLKLUGDGXCTVJSQ, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 5.161.89.252:56852->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432961023 (510h2m55.785746131s) [noise=true , address=45.91.116.5:4297 , piece_id=BCSTXIIO2MSEGGVGOJN3DQ26LF5M42TKOPX5RSU7GLVKJUXVBNWQ, seg_pos={0 0}, upload=448970]: Done uploading piece. err: ecclient: upload failed (node:12pba8P3R4hb1xBifPDXzMKXuWgwwpPCx4MXPuBDU7Nupj2LKLW, address:45.91.116.5:4297): protocol: read tcp 5.161.89.252:18170->45.91.116.5:4297: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:11:20.432979528 (510h2m55.785764636s) [seg_pos={0 0}, upload=448970]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Sep 19 20:11:20.449392930 (510h2m55.802178029s) [upload=448970]: Done uploading segments... Sep 19 20:11:20.449408840 (510h2m55.802193948s) [upload=448970]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.01765716s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X..X.] duration: 11.348426872s FAIL (2025-09-19 20:10:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 8953]: Uploading piece... Sep 19 20:09:47.612742562 (510h1m22.96552766s) [noise=true , address=145.79.227.5:4222 , piece_id=SAMPLFPUUKP4RXUXXNLE6D3VZ3HQSTH7ZTFACW6EE5D3OTXAIPQQ, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.612806251 (510h1m22.965591349s) [noise=true , address=77.90.31.19:4842 , piece_id=WH4SSAVRQXW2YLKCLJKMHKGQHQO3Q6H6LDRZFR33R6Y522ZP34QA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613043837 (510h1m22.965828945s) [noise=true , address=77.90.31.10:4517 , piece_id=M73PDMFC4QZUZVF5BAFV75JL5ULM53RZSOL4HXFGB5I57SAMELRA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613050890 (510h1m22.965835999s) [noise=true , address=45.91.116.11:4117 , piece_id=AOXBA2ZFHMZL7P3AXGLGS2VCEL44X7M6QUV7XBJFZOFVTICCNSAA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613254042 (510h1m22.96603914s) [noise=true , address=77.90.31.19:4841 , piece_id=HWFU6IW42YLFMQSMTQOYQGXWTT7PKMHK4ZNAQSUJM7IG6YWDVY4A, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613269141 (510h1m22.966054239s) [noise=true , address=145.79.227.13:4802 , piece_id=XRNN76GW5FPKQFMPW3WUBZ2KWPTYVWOSMALTJ5QW63SGU4LXHJ3Q, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613533055 (510h1m22.966318154s) [noise=true , address=77.90.31.11:4553 , piece_id=RP6L2RWNHO7QNQ7IWBHN6CSIG5CDMJL5SAUIY5BAEQ6DUHBB7ZIA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613745905 (510h1m22.966531013s) [noise=true , address=45.91.116.10:4083 , piece_id=TK35WAKJDW6EUNMPPDZKY4H4SNY42CWAZXYI5UE2424N5DCUCINA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613953644 (510h1m22.966738752s) [noise=true , address=77.90.31.12:4590 , piece_id=LA3D7LYKB53SXCTIY4UBCDN6W6W673NVB3N4H3EPQCSRFEBFSQCQ, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.614200769 (510h1m22.966985877s) [noise=true , address=45.91.116.14:4189 , piece_id=CVIFTRRISH3G6PFAIBUMNXEWLWTUERCCP4G2SMBCMR7A7DEBTAAA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.614412586 (510h1m22.967197684s) [noise=true , address=145.79.227.16:4912 , piece_id=IPVXKJDJZLDL4XBCL3FKKK2PM2GJQNUOC5MTKP7FP46SCWAPN7ZQ, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.614837984 (510h1m22.967623092s) [noise=true , address=45.91.116.4:4940 , piece_id=JKBNREL5TJZX3DMI77NDESM2FWTZA5GWT6JZQ4LA2WSA4MG35UXA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.616299477 (510h1m22.969084585s) [noise=true , address=77.90.31.17:4772 , piece_id=R23DL7GAHVRCTEVLAADIQ7MIVDNKGIX6IXTYKQP3TNKPBYOURBGQ, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.617588045 (510h1m22.970373153s) [noise=true , address=77.90.31.21:4879 , piece_id=35ZQVM2GBMQTQETDZMHMBOXSCGSQXLSUUJNWT7T3GPDQVUNYSKPA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.674824115 (510h1m23.027609223s) [upload=448953]: Next returned nil segment Sep 19 20:09:47.674838933 (510h1m23.027624051s) [upload=448953]: Waiting for error group managing segments... Sep 19 20:09:47.682444807 (510h1m23.035229915s) [noise=true , address=145.79.227.13:4802 , piece_id=XRNN76GW5FPKQFMPW3WUBZ2KWPTYVWOSMALTJ5QW63SGU4LXHJ3Q, seg_pos={0 0}, upload=448953]: Done uploading piece. err: <nil> Sep 19 20:09:47.684807551 (510h1m23.037592659s) [noise=true , address=145.79.227.12:4767 , piece_id=DLGSW4KU6SNDC5SE2SG3JPFKKGYU7QYDPBVFQABHEUF5T4I4Y24Q, seg_pos={0 0}, upload=448953]: Done uploading piece. err: <nil> Sep 19 20:09:47.856468343 (510h1m23.209253471s) [noise=true , address=145.79.227.16:4912 , piece_id=IPVXKJDJZLDL4XBCL3FKKK2PM2GJQNUOC5MTKP7FP46SCWAPN7ZQ, seg_pos={0 0}, upload=448953]: Done uploading piece. err: <nil> Sep 19 20:09:48.486442816 (510h1m23.839227954s) [noise=true , address=145.79.227.5:4222 , piece_id=SAMPLFPUUKP4RXUXXNLE6D3VZ3HQSTH7ZTFACW6EE5D3OTXAIPQQ, seg_pos={0 0}, upload=448953]: Done uploading piece. err: <nil> Sep 19 20:10:17.569546263 (510h1m52.922331361s) [noise=true , address=45.91.116.4:4940 , piece_id=JKBNREL5TJZX3DMI77NDESM2FWTZA5GWT6JZQ4LA2WSA4MG35UXA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569645730 (510h1m52.922430827s) [noise=true , address=77.90.31.21:4879 , piece_id=35ZQVM2GBMQTQETDZMHMBOXSCGSQXLSUUJNWT7T3GPDQVUNYSKPA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569677459 (510h1m52.922462567s) [noise=true , address=77.90.31.19:4842 , piece_id=WH4SSAVRQXW2YLKCLJKMHKGQHQO3Q6H6LDRZFR33R6Y522ZP34QA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569709219 (510h1m52.922494327s) [noise=true , address=77.90.31.19:4841 , piece_id=HWFU6IW42YLFMQSMTQOYQGXWTT7PKMHK4ZNAQSUJM7IG6YWDVY4A, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:36748->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569758501 (510h1m52.922543609s) [noise=true , address=45.91.116.14:4189 , piece_id=CVIFTRRISH3G6PFAIBUMNXEWLWTUERCCP4G2SMBCMR7A7DEBTAAA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:24272->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569773759 (510h1m52.922558867s) [noise=true , address=45.91.116.10:4083 , piece_id=TK35WAKJDW6EUNMPPDZKY4H4SNY42CWAZXYI5UE2424N5DCUCINA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 5.161.89.252:55846->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569795981 (510h1m52.922581089s) [noise=true , address=45.91.116.8:4045 , piece_id=X26U7ENFOXSCPOFQTYZQ3LRN4SAKGYGSJXMYSW2PM5WOOZL67LXA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 5.161.89.252:55428->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569809627 (510h1m52.922594725s) [noise=true , address=77.90.31.17:4772 , piece_id=R23DL7GAHVRCTEVLAADIQ7MIVDNKGIX6IXTYKQP3TNKPBYOURBGQ, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 5.161.89.252:54254->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569832179 (510h1m52.922617287s) [noise=true , address=45.91.116.11:4117 , piece_id=AOXBA2ZFHMZL7P3AXGLGS2VCEL44X7M6QUV7XBJFZOFVTICCNSAA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 5.161.89.252:29378->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569842739 (510h1m52.922627847s) [noise=true , address=77.90.31.10:4517 , piece_id=M73PDMFC4QZUZVF5BAFV75JL5ULM53RZSOL4HXFGB5I57SAMELRA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: read tcp 5.161.89.252:48536->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569874318 (510h1m52.922659427s) [noise=true , address=77.90.31.11:4553 , piece_id=RP6L2RWNHO7QNQ7IWBHN6CSIG5CDMJL5SAUIY5BAEQ6DUHBB7ZIA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:14180->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569878887 (510h1m52.922663995s) [noise=true , address=77.90.31.12:4590 , piece_id=LA3D7LYKB53SXCTIY4UBCDN6W6W673NVB3N4H3EPQCSRFEBFSQCQ, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 5.161.89.252:21884->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569902291 (510h1m52.922687399s) [seg_pos={0 0}, upload=448953]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Sep 19 20:10:17.594296509 (510h1m52.947081607s) [upload=448953]: Done uploading segments... Sep 19 20:10:17.594313240 (510h1m52.947098349s) [upload=448953]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 11.348448061s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.000942667s FAIL (2025-09-19 20:10:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 30.000962725s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................X..X] duration: 30.026250637s FAIL (2025-09-19 20:10:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 4) ======LOGS====== 8953]: Uploading piece... Sep 19 20:09:47.612742562 (510h1m22.96552766s) [noise=true , address=145.79.227.5:4222 , piece_id=SAMPLFPUUKP4RXUXXNLE6D3VZ3HQSTH7ZTFACW6EE5D3OTXAIPQQ, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.612806251 (510h1m22.965591349s) [noise=true , address=77.90.31.19:4842 , piece_id=WH4SSAVRQXW2YLKCLJKMHKGQHQO3Q6H6LDRZFR33R6Y522ZP34QA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613043837 (510h1m22.965828945s) [noise=true , address=77.90.31.10:4517 , piece_id=M73PDMFC4QZUZVF5BAFV75JL5ULM53RZSOL4HXFGB5I57SAMELRA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613050890 (510h1m22.965835999s) [noise=true , address=45.91.116.11:4117 , piece_id=AOXBA2ZFHMZL7P3AXGLGS2VCEL44X7M6QUV7XBJFZOFVTICCNSAA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613254042 (510h1m22.96603914s) [noise=true , address=77.90.31.19:4841 , piece_id=HWFU6IW42YLFMQSMTQOYQGXWTT7PKMHK4ZNAQSUJM7IG6YWDVY4A, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613269141 (510h1m22.966054239s) [noise=true , address=145.79.227.13:4802 , piece_id=XRNN76GW5FPKQFMPW3WUBZ2KWPTYVWOSMALTJ5QW63SGU4LXHJ3Q, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613533055 (510h1m22.966318154s) [noise=true , address=77.90.31.11:4553 , piece_id=RP6L2RWNHO7QNQ7IWBHN6CSIG5CDMJL5SAUIY5BAEQ6DUHBB7ZIA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613745905 (510h1m22.966531013s) [noise=true , address=45.91.116.10:4083 , piece_id=TK35WAKJDW6EUNMPPDZKY4H4SNY42CWAZXYI5UE2424N5DCUCINA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.613953644 (510h1m22.966738752s) [noise=true , address=77.90.31.12:4590 , piece_id=LA3D7LYKB53SXCTIY4UBCDN6W6W673NVB3N4H3EPQCSRFEBFSQCQ, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.614200769 (510h1m22.966985877s) [noise=true , address=45.91.116.14:4189 , piece_id=CVIFTRRISH3G6PFAIBUMNXEWLWTUERCCP4G2SMBCMR7A7DEBTAAA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.614412586 (510h1m22.967197684s) [noise=true , address=145.79.227.16:4912 , piece_id=IPVXKJDJZLDL4XBCL3FKKK2PM2GJQNUOC5MTKP7FP46SCWAPN7ZQ, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.614837984 (510h1m22.967623092s) [noise=true , address=45.91.116.4:4940 , piece_id=JKBNREL5TJZX3DMI77NDESM2FWTZA5GWT6JZQ4LA2WSA4MG35UXA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.616299477 (510h1m22.969084585s) [noise=true , address=77.90.31.17:4772 , piece_id=R23DL7GAHVRCTEVLAADIQ7MIVDNKGIX6IXTYKQP3TNKPBYOURBGQ, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.617588045 (510h1m22.970373153s) [noise=true , address=77.90.31.21:4879 , piece_id=35ZQVM2GBMQTQETDZMHMBOXSCGSQXLSUUJNWT7T3GPDQVUNYSKPA, seg_pos={0 0}, upload=448953]: Uploading piece... Sep 19 20:09:47.674824115 (510h1m23.027609223s) [upload=448953]: Next returned nil segment Sep 19 20:09:47.674838933 (510h1m23.027624051s) [upload=448953]: Waiting for error group managing segments... Sep 19 20:09:47.682444807 (510h1m23.035229915s) [noise=true , address=145.79.227.13:4802 , piece_id=XRNN76GW5FPKQFMPW3WUBZ2KWPTYVWOSMALTJ5QW63SGU4LXHJ3Q, seg_pos={0 0}, upload=448953]: Done uploading piece. err: <nil> Sep 19 20:09:47.684807551 (510h1m23.037592659s) [noise=true , address=145.79.227.12:4767 , piece_id=DLGSW4KU6SNDC5SE2SG3JPFKKGYU7QYDPBVFQABHEUF5T4I4Y24Q, seg_pos={0 0}, upload=448953]: Done uploading piece. err: <nil> Sep 19 20:09:47.856468343 (510h1m23.209253471s) [noise=true , address=145.79.227.16:4912 , piece_id=IPVXKJDJZLDL4XBCL3FKKK2PM2GJQNUOC5MTKP7FP46SCWAPN7ZQ, seg_pos={0 0}, upload=448953]: Done uploading piece. err: <nil> Sep 19 20:09:48.486442816 (510h1m23.839227954s) [noise=true , address=145.79.227.5:4222 , piece_id=SAMPLFPUUKP4RXUXXNLE6D3VZ3HQSTH7ZTFACW6EE5D3OTXAIPQQ, seg_pos={0 0}, upload=448953]: Done uploading piece. err: <nil> Sep 19 20:10:17.569546263 (510h1m52.922331361s) [noise=true , address=45.91.116.4:4940 , piece_id=JKBNREL5TJZX3DMI77NDESM2FWTZA5GWT6JZQ4LA2WSA4MG35UXA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe, address:45.91.116.4:4940): protocol: rpc: dial tcp 45.91.116.4:4940: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569645730 (510h1m52.922430827s) [noise=true , address=77.90.31.21:4879 , piece_id=35ZQVM2GBMQTQETDZMHMBOXSCGSQXLSUUJNWT7T3GPDQVUNYSKPA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, address:77.90.31.21:4879): protocol: rpc: dial tcp 77.90.31.21:4879: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569677459 (510h1m52.922462567s) [noise=true , address=77.90.31.19:4842 , piece_id=WH4SSAVRQXW2YLKCLJKMHKGQHQO3Q6H6LDRZFR33R6Y522ZP34QA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3, address:77.90.31.19:4842): protocol: rpc: dial tcp 77.90.31.19:4842: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569709219 (510h1m52.922494327s) [noise=true , address=77.90.31.19:4841 , piece_id=HWFU6IW42YLFMQSMTQOYQGXWTT7PKMHK4ZNAQSUJM7IG6YWDVY4A, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: read tcp 5.161.89.252:36748->77.90.31.19:4841: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569758501 (510h1m52.922543609s) [noise=true , address=45.91.116.14:4189 , piece_id=CVIFTRRISH3G6PFAIBUMNXEWLWTUERCCP4G2SMBCMR7A7DEBTAAA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N, address:45.91.116.14:4189): protocol: read tcp 5.161.89.252:24272->45.91.116.14:4189: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569773759 (510h1m52.922558867s) [noise=true , address=45.91.116.10:4083 , piece_id=TK35WAKJDW6EUNMPPDZKY4H4SNY42CWAZXYI5UE2424N5DCUCINA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: read tcp 5.161.89.252:55846->45.91.116.10:4083: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569795981 (510h1m52.922581089s) [noise=true , address=45.91.116.8:4045 , piece_id=X26U7ENFOXSCPOFQTYZQ3LRN4SAKGYGSJXMYSW2PM5WOOZL67LXA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: read tcp 5.161.89.252:55428->45.91.116.8:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569809627 (510h1m52.922594725s) [noise=true , address=77.90.31.17:4772 , piece_id=R23DL7GAHVRCTEVLAADIQ7MIVDNKGIX6IXTYKQP3TNKPBYOURBGQ, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:18CNfMYrF1Gqgabi62fG23CagDZHW2QyZZNYKKTSjWQKLS4ppE, address:77.90.31.17:4772): protocol: read tcp 5.161.89.252:54254->77.90.31.17:4772: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569832179 (510h1m52.922617287s) [noise=true , address=45.91.116.11:4117 , piece_id=AOXBA2ZFHMZL7P3AXGLGS2VCEL44X7M6QUV7XBJFZOFVTICCNSAA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:167JkPLSCrURmtr6PmVHr4i6huBvTzVXXxhLjkB1RkfC6aqox6, address:45.91.116.11:4117): protocol: read tcp 5.161.89.252:29378->45.91.116.11:4117: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569842739 (510h1m52.922627847s) [noise=true , address=77.90.31.10:4517 , piece_id=M73PDMFC4QZUZVF5BAFV75JL5ULM53RZSOL4HXFGB5I57SAMELRA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A, address:77.90.31.10:4517): protocol: read tcp 5.161.89.252:48536->77.90.31.10:4517: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569874318 (510h1m52.922659427s) [noise=true , address=77.90.31.11:4553 , piece_id=RP6L2RWNHO7QNQ7IWBHN6CSIG5CDMJL5SAUIY5BAEQ6DUHBB7ZIA, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, address:77.90.31.11:4553): protocol: read tcp 5.161.89.252:14180->77.90.31.11:4553: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569878887 (510h1m52.922663995s) [noise=true , address=77.90.31.12:4590 , piece_id=LA3D7LYKB53SXCTIY4UBCDN6W6W673NVB3N4H3EPQCSRFEBFSQCQ, seg_pos={0 0}, upload=448953]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: read tcp 5.161.89.252:21884->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:10:17.569902291 (510h1m52.922687399s) [seg_pos={0 0}, upload=448953]: Done waiting for segment. successful: 4 optimal: 13 errs: <nil> Sep 19 20:10:17.594296509 (510h1m52.947081607s) [upload=448953]: Done uploading segments... Sep 19 20:10:17.594313240 (510h1m52.947098349s) [upload=448953]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 4) Duration: 30.026266847s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 1.364785184s FAIL (2025-09-19 20:08:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 19 20:07:32.774345835 (509h59m8.127130933s) [noise=true , address=45.91.116.6:4973 , piece_id=5LNGCULVMYBTP7XI3MR2R5DRHP5SUEPHMA5Y7Q5UHQ2FKH4IOF4Q, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774416417 (509h59m8.127201525s) [noise=true , address=45.91.116.10:4083 , piece_id=U33CY6NZ66RKVNKQOPY6Y2S7DPGS77NH3CXDWCOOY5U2QX5NVU6A, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774652882 (509h59m8.12743799s) [noise=true , address=45.91.116.5:4299 , piece_id=EKF43QQITQAFGDZYMGUWBMROXDQFCUNLOO3TNTW7FXSPW6526NKQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774804366 (509h59m8.127589474s) [noise=true , address=45.91.116.14:4190 , piece_id=RFZ43GBV5IWZ3XI3T37A7XZ5SOW75ABHP7P34UKG4DBN7VO63I2Q, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774867705 (509h59m8.127652813s) [noise=true , address=77.90.31.16:4733 , piece_id=4FWGFGBKBXPB7HGUWJUNLZDOUSXPBPME246WBSPGRQVTR3OAUI6A, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775076607 (509h59m8.127861715s) [noise=true , address=77.90.31.13:4628 , piece_id=5RPRXMWNHEJUHUAOJAO3UQIC7HUDBLLPEYBLTPYP7CBM4AAMETPQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775132611 (509h59m8.12791771s) [noise=true , address=145.79.227.17:4971 , piece_id=5EXYCHVEJRO7SGDTQSFISPY6QAKSASC7QM3EPAYP2HMKVNR5UCMQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775288765 (509h59m8.128073873s) [noise=true , address=45.91.116.5:4298 , piece_id=YBFPQ7DLTREC7REDG7R2UB6DWELMLEZNZFOWSD5YARJAGWGDSQNQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775468132 (509h59m8.12825324s) [noise=true , address=77.90.31.15:4697 , piece_id=CIZ4AWIN3FCTWJSXTLQ6LGAQQ3T5PB7RM7UHEUKVDMLHMZDEXCHQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775535859 (509h59m8.128320967s) [noise=true , address=45.91.116.13:4154 , piece_id=4TPETKLGS7W3TFAOSQE7XLPRVZNZD2JHUOYMHMIQKNNCYCPZ5SPQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775759368 (509h59m8.128544466s) [noise=true , address=45.91.116.9:4262 , piece_id=4ONMQYLAFCX7VGE7IPXHLE6IW2SQCXAJAEFSIBNMBDFXHH7DRWOA, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775810083 (509h59m8.128595191s) [noise=true , address=45.91.116.4:4937 , piece_id=K6HXTZXJDPNZMRYKL65EYPJQ5LO4HUMXQXVHLRP3HKCSBOZKB2UQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776025076 (509h59m8.128810184s) [noise=true , address=77.90.31.21:4879 , piece_id=FSTO3T5B2EKFTPTEZIE53PUMUUDKOZKQAASA3HRWWQ7O5CQ6PFFA, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776145392 (509h59m8.1289305s) [noise=true , address=45.91.116.8:4045 , piece_id=6LRPPHPHVRYUFMIIUYTTO2CL5WRRMSWJASMJ32ROTJZIBVGMZHLA, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776240902 (509h59m8.129026s) [noise=true , address=145.79.227.7:4295 , piece_id=QYQFLNNDK5GOOLFLMLBXZEPUZLMPRYLW2UJEVBT6NLLVYCYGTPVQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776481944 (509h59m8.129267042s) [noise=true , address=145.79.227.13:4802 , piece_id=7XS33F3ZU3BUUVCXUWU3OQ7AUTDF6BA63DFGMTH5ZDKF67GMAKRQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.849041962 (509h59m8.20182708s) [upload=448920]: Next returned nil segment Sep 19 20:07:32.849056960 (509h59m8.201842059s) [upload=448920]: Waiting for error group managing segments... Sep 19 20:07:32.856512782 (509h59m8.20929789s) [noise=true , address=145.79.227.17:4971 , piece_id=5EXYCHVEJRO7SGDTQSFISPY6QAKSASC7QM3EPAYP2HMKVNR5UCMQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:32.857097799 (509h59m8.209882907s) [noise=true , address=145.79.227.7:4295 , piece_id=QYQFLNNDK5GOOLFLMLBXZEPUZLMPRYLW2UJEVBT6NLLVYCYGTPVQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:32.859056726 (509h59m8.211841844s) [noise=true , address=145.79.227.13:4802 , piece_id=7XS33F3ZU3BUUVCXUWU3OQ7AUTDF6BA63DFGMTH5ZDKF67GMAKRQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:35.782241505 (509h59m11.135026613s) [noise=true , address=77.90.31.21:4879 , piece_id=FSTO3T5B2EKFTPTEZIE53PUMUUDKOZKQAASA3HRWWQ7O5CQ6PFFA, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:37.016800765 (509h59m12.369585873s) [noise=true , address=77.90.31.16:4733 , piece_id=4FWGFGBKBXPB7HGUWJUNLZDOUSXPBPME246WBSPGRQVTR3OAUI6A, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:37.467822321 (509h59m12.820607459s) [noise=true , address=45.91.116.5:4299 , piece_id=EKF43QQITQAFGDZYMGUWBMROXDQFCUNLOO3TNTW7FXSPW6526NKQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:37.907472168 (509h59m13.260257306s) [noise=true , address=45.91.116.9:4262 , piece_id=4ONMQYLAFCX7VGE7IPXHLE6IW2SQCXAJAEFSIBNMBDFXHH7DRWOA, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:39.477854571 (509h59m14.830639699s) [noise=true , address=45.91.116.14:4190 , piece_id=RFZ43GBV5IWZ3XI3T37A7XZ5SOW75ABHP7P34UKG4DBN7VO63I2Q, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:42.773785732 (509h59m18.12657086s) [noise=true , address=45.91.116.4:4937 , piece_id=K6HXTZXJDPNZMRYKL65EYPJQ5LO4HUMXQXVHLRP3HKCSBOZKB2UQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:08:02.713152052 (509h59m38.065937149s) [noise=true , address=45.91.116.10:4083 , piece_id=U33CY6NZ66RKVNKQOPY6Y2S7DPGS77NH3CXDWCOOY5U2QX5NVU6A, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 5.161.89.252:52360->45.91.116.10:4083: use of closed network connection; write tcp 5.161.89.252:52360->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:52360->45.91.116.10:4083: use of closed network connection Sep 19 20:08:02.713240237 (509h59m38.066025335s) [noise=true , address=45.91.116.13:4154 , piece_id=4TPETKLGS7W3TFAOSQE7XLPRVZNZD2JHUOYMHMIQKNNCYCPZ5SPQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 5.161.89.252:33252->45.91.116.13:4154: use of closed network connection; write tcp 5.161.89.252:33252->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:33252->45.91.116.13:4154: use of closed network connection Sep 19 20:08:02.713300701 (509h59m38.0660858s) [noise=true , address=45.91.116.8:4045 , piece_id=6LRPPHPHVRYUFMIIUYTTO2CL5WRRMSWJASMJ32ROTJZIBVGMZHLA, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:51966->45.91.116.8:4045: use of closed network connection; write tcp 5.161.89.252:51966->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:51966->45.91.116.8:4045: use of closed network connection Sep 19 20:08:02.713319155 (509h59m38.066104264s) [noise=true , address=45.91.116.6:4973 , piece_id=5LNGCULVMYBTP7XI3MR2R5DRHP5SUEPHMA5Y7Q5UHQ2FKH4IOF4Q, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:16098->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:08:02.713455451 (509h59m38.066240559s) [noise=true , address=45.91.116.5:4298 , piece_id=YBFPQ7DLTREC7REDG7R2UB6DWELMLEZNZFOWSD5YARJAGWGDSQNQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 5.161.89.252:61614->45.91.116.5:4298: use of closed network connection; write tcp 5.161.89.252:61614->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61614->45.91.116.5:4298: use of closed network connection Sep 19 20:08:02.713521586 (509h59m38.066306694s) [noise=true , address=77.90.31.13:4628 , piece_id=5RPRXMWNHEJUHUAOJAO3UQIC7HUDBLLPEYBLTPYP7CBM4AAMETPQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 5.161.89.252:23444->77.90.31.13:4628: use of closed network connection; write tcp 5.161.89.252:23444->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:23444->77.90.31.13:4628: use of closed network connection Sep 19 20:08:02.713573042 (509h59m38.06635815s) [noise=true , address=77.90.31.15:4697 , piece_id=CIZ4AWIN3FCTWJSXTLQ6LGAQQ3T5PB7RM7UHEUKVDMLHMZDEXCHQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 5.161.89.252:33274->77.90.31.15:4697: use of closed network connection; write tcp 5.161.89.252:33274->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:33274->77.90.31.15:4697: use of closed network connection Sep 19 20:08:02.713591537 (509h59m38.066376645s) [seg_pos={0 0}, upload=448920]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 19 20:08:02.735261062 (509h59m38.08804617s) [upload=448920]: Done uploading segments... Sep 19 20:08:02.735275329 (509h59m38.088060427s) [upload=448920]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 1.364808319s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.504167559s FAIL (2025-09-19 20:08:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 19 20:07:32.774345835 (509h59m8.127130933s) [noise=true , address=45.91.116.6:4973 , piece_id=5LNGCULVMYBTP7XI3MR2R5DRHP5SUEPHMA5Y7Q5UHQ2FKH4IOF4Q, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774416417 (509h59m8.127201525s) [noise=true , address=45.91.116.10:4083 , piece_id=U33CY6NZ66RKVNKQOPY6Y2S7DPGS77NH3CXDWCOOY5U2QX5NVU6A, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774652882 (509h59m8.12743799s) [noise=true , address=45.91.116.5:4299 , piece_id=EKF43QQITQAFGDZYMGUWBMROXDQFCUNLOO3TNTW7FXSPW6526NKQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774804366 (509h59m8.127589474s) [noise=true , address=45.91.116.14:4190 , piece_id=RFZ43GBV5IWZ3XI3T37A7XZ5SOW75ABHP7P34UKG4DBN7VO63I2Q, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774867705 (509h59m8.127652813s) [noise=true , address=77.90.31.16:4733 , piece_id=4FWGFGBKBXPB7HGUWJUNLZDOUSXPBPME246WBSPGRQVTR3OAUI6A, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775076607 (509h59m8.127861715s) [noise=true , address=77.90.31.13:4628 , piece_id=5RPRXMWNHEJUHUAOJAO3UQIC7HUDBLLPEYBLTPYP7CBM4AAMETPQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775132611 (509h59m8.12791771s) [noise=true , address=145.79.227.17:4971 , piece_id=5EXYCHVEJRO7SGDTQSFISPY6QAKSASC7QM3EPAYP2HMKVNR5UCMQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775288765 (509h59m8.128073873s) [noise=true , address=45.91.116.5:4298 , piece_id=YBFPQ7DLTREC7REDG7R2UB6DWELMLEZNZFOWSD5YARJAGWGDSQNQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775468132 (509h59m8.12825324s) [noise=true , address=77.90.31.15:4697 , piece_id=CIZ4AWIN3FCTWJSXTLQ6LGAQQ3T5PB7RM7UHEUKVDMLHMZDEXCHQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775535859 (509h59m8.128320967s) [noise=true , address=45.91.116.13:4154 , piece_id=4TPETKLGS7W3TFAOSQE7XLPRVZNZD2JHUOYMHMIQKNNCYCPZ5SPQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775759368 (509h59m8.128544466s) [noise=true , address=45.91.116.9:4262 , piece_id=4ONMQYLAFCX7VGE7IPXHLE6IW2SQCXAJAEFSIBNMBDFXHH7DRWOA, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775810083 (509h59m8.128595191s) [noise=true , address=45.91.116.4:4937 , piece_id=K6HXTZXJDPNZMRYKL65EYPJQ5LO4HUMXQXVHLRP3HKCSBOZKB2UQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776025076 (509h59m8.128810184s) [noise=true , address=77.90.31.21:4879 , piece_id=FSTO3T5B2EKFTPTEZIE53PUMUUDKOZKQAASA3HRWWQ7O5CQ6PFFA, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776145392 (509h59m8.1289305s) [noise=true , address=45.91.116.8:4045 , piece_id=6LRPPHPHVRYUFMIIUYTTO2CL5WRRMSWJASMJ32ROTJZIBVGMZHLA, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776240902 (509h59m8.129026s) [noise=true , address=145.79.227.7:4295 , piece_id=QYQFLNNDK5GOOLFLMLBXZEPUZLMPRYLW2UJEVBT6NLLVYCYGTPVQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776481944 (509h59m8.129267042s) [noise=true , address=145.79.227.13:4802 , piece_id=7XS33F3ZU3BUUVCXUWU3OQ7AUTDF6BA63DFGMTH5ZDKF67GMAKRQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.849041962 (509h59m8.20182708s) [upload=448920]: Next returned nil segment Sep 19 20:07:32.849056960 (509h59m8.201842059s) [upload=448920]: Waiting for error group managing segments... Sep 19 20:07:32.856512782 (509h59m8.20929789s) [noise=true , address=145.79.227.17:4971 , piece_id=5EXYCHVEJRO7SGDTQSFISPY6QAKSASC7QM3EPAYP2HMKVNR5UCMQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:32.857097799 (509h59m8.209882907s) [noise=true , address=145.79.227.7:4295 , piece_id=QYQFLNNDK5GOOLFLMLBXZEPUZLMPRYLW2UJEVBT6NLLVYCYGTPVQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:32.859056726 (509h59m8.211841844s) [noise=true , address=145.79.227.13:4802 , piece_id=7XS33F3ZU3BUUVCXUWU3OQ7AUTDF6BA63DFGMTH5ZDKF67GMAKRQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:35.782241505 (509h59m11.135026613s) [noise=true , address=77.90.31.21:4879 , piece_id=FSTO3T5B2EKFTPTEZIE53PUMUUDKOZKQAASA3HRWWQ7O5CQ6PFFA, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:37.016800765 (509h59m12.369585873s) [noise=true , address=77.90.31.16:4733 , piece_id=4FWGFGBKBXPB7HGUWJUNLZDOUSXPBPME246WBSPGRQVTR3OAUI6A, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:37.467822321 (509h59m12.820607459s) [noise=true , address=45.91.116.5:4299 , piece_id=EKF43QQITQAFGDZYMGUWBMROXDQFCUNLOO3TNTW7FXSPW6526NKQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:37.907472168 (509h59m13.260257306s) [noise=true , address=45.91.116.9:4262 , piece_id=4ONMQYLAFCX7VGE7IPXHLE6IW2SQCXAJAEFSIBNMBDFXHH7DRWOA, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:39.477854571 (509h59m14.830639699s) [noise=true , address=45.91.116.14:4190 , piece_id=RFZ43GBV5IWZ3XI3T37A7XZ5SOW75ABHP7P34UKG4DBN7VO63I2Q, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:42.773785732 (509h59m18.12657086s) [noise=true , address=45.91.116.4:4937 , piece_id=K6HXTZXJDPNZMRYKL65EYPJQ5LO4HUMXQXVHLRP3HKCSBOZKB2UQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:08:02.713152052 (509h59m38.065937149s) [noise=true , address=45.91.116.10:4083 , piece_id=U33CY6NZ66RKVNKQOPY6Y2S7DPGS77NH3CXDWCOOY5U2QX5NVU6A, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 5.161.89.252:52360->45.91.116.10:4083: use of closed network connection; write tcp 5.161.89.252:52360->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:52360->45.91.116.10:4083: use of closed network connection Sep 19 20:08:02.713240237 (509h59m38.066025335s) [noise=true , address=45.91.116.13:4154 , piece_id=4TPETKLGS7W3TFAOSQE7XLPRVZNZD2JHUOYMHMIQKNNCYCPZ5SPQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 5.161.89.252:33252->45.91.116.13:4154: use of closed network connection; write tcp 5.161.89.252:33252->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:33252->45.91.116.13:4154: use of closed network connection Sep 19 20:08:02.713300701 (509h59m38.0660858s) [noise=true , address=45.91.116.8:4045 , piece_id=6LRPPHPHVRYUFMIIUYTTO2CL5WRRMSWJASMJ32ROTJZIBVGMZHLA, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:51966->45.91.116.8:4045: use of closed network connection; write tcp 5.161.89.252:51966->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:51966->45.91.116.8:4045: use of closed network connection Sep 19 20:08:02.713319155 (509h59m38.066104264s) [noise=true , address=45.91.116.6:4973 , piece_id=5LNGCULVMYBTP7XI3MR2R5DRHP5SUEPHMA5Y7Q5UHQ2FKH4IOF4Q, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:16098->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:08:02.713455451 (509h59m38.066240559s) [noise=true , address=45.91.116.5:4298 , piece_id=YBFPQ7DLTREC7REDG7R2UB6DWELMLEZNZFOWSD5YARJAGWGDSQNQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 5.161.89.252:61614->45.91.116.5:4298: use of closed network connection; write tcp 5.161.89.252:61614->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61614->45.91.116.5:4298: use of closed network connection Sep 19 20:08:02.713521586 (509h59m38.066306694s) [noise=true , address=77.90.31.13:4628 , piece_id=5RPRXMWNHEJUHUAOJAO3UQIC7HUDBLLPEYBLTPYP7CBM4AAMETPQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 5.161.89.252:23444->77.90.31.13:4628: use of closed network connection; write tcp 5.161.89.252:23444->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:23444->77.90.31.13:4628: use of closed network connection Sep 19 20:08:02.713573042 (509h59m38.06635815s) [noise=true , address=77.90.31.15:4697 , piece_id=CIZ4AWIN3FCTWJSXTLQ6LGAQQ3T5PB7RM7UHEUKVDMLHMZDEXCHQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 5.161.89.252:33274->77.90.31.15:4697: use of closed network connection; write tcp 5.161.89.252:33274->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:33274->77.90.31.15:4697: use of closed network connection Sep 19 20:08:02.713591537 (509h59m38.066376645s) [seg_pos={0 0}, upload=448920]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 19 20:08:02.735261062 (509h59m38.08804617s) [upload=448920]: Done uploading segments... Sep 19 20:08:02.735275329 (509h59m38.088060427s) [upload=448920]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 5.50418862s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.023324245s FAIL (2025-09-19 20:08:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Sep 19 20:07:32.774345835 (509h59m8.127130933s) [noise=true , address=45.91.116.6:4973 , piece_id=5LNGCULVMYBTP7XI3MR2R5DRHP5SUEPHMA5Y7Q5UHQ2FKH4IOF4Q, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774416417 (509h59m8.127201525s) [noise=true , address=45.91.116.10:4083 , piece_id=U33CY6NZ66RKVNKQOPY6Y2S7DPGS77NH3CXDWCOOY5U2QX5NVU6A, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774652882 (509h59m8.12743799s) [noise=true , address=45.91.116.5:4299 , piece_id=EKF43QQITQAFGDZYMGUWBMROXDQFCUNLOO3TNTW7FXSPW6526NKQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774804366 (509h59m8.127589474s) [noise=true , address=45.91.116.14:4190 , piece_id=RFZ43GBV5IWZ3XI3T37A7XZ5SOW75ABHP7P34UKG4DBN7VO63I2Q, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.774867705 (509h59m8.127652813s) [noise=true , address=77.90.31.16:4733 , piece_id=4FWGFGBKBXPB7HGUWJUNLZDOUSXPBPME246WBSPGRQVTR3OAUI6A, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775076607 (509h59m8.127861715s) [noise=true , address=77.90.31.13:4628 , piece_id=5RPRXMWNHEJUHUAOJAO3UQIC7HUDBLLPEYBLTPYP7CBM4AAMETPQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775132611 (509h59m8.12791771s) [noise=true , address=145.79.227.17:4971 , piece_id=5EXYCHVEJRO7SGDTQSFISPY6QAKSASC7QM3EPAYP2HMKVNR5UCMQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775288765 (509h59m8.128073873s) [noise=true , address=45.91.116.5:4298 , piece_id=YBFPQ7DLTREC7REDG7R2UB6DWELMLEZNZFOWSD5YARJAGWGDSQNQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775468132 (509h59m8.12825324s) [noise=true , address=77.90.31.15:4697 , piece_id=CIZ4AWIN3FCTWJSXTLQ6LGAQQ3T5PB7RM7UHEUKVDMLHMZDEXCHQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775535859 (509h59m8.128320967s) [noise=true , address=45.91.116.13:4154 , piece_id=4TPETKLGS7W3TFAOSQE7XLPRVZNZD2JHUOYMHMIQKNNCYCPZ5SPQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775759368 (509h59m8.128544466s) [noise=true , address=45.91.116.9:4262 , piece_id=4ONMQYLAFCX7VGE7IPXHLE6IW2SQCXAJAEFSIBNMBDFXHH7DRWOA, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.775810083 (509h59m8.128595191s) [noise=true , address=45.91.116.4:4937 , piece_id=K6HXTZXJDPNZMRYKL65EYPJQ5LO4HUMXQXVHLRP3HKCSBOZKB2UQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776025076 (509h59m8.128810184s) [noise=true , address=77.90.31.21:4879 , piece_id=FSTO3T5B2EKFTPTEZIE53PUMUUDKOZKQAASA3HRWWQ7O5CQ6PFFA, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776145392 (509h59m8.1289305s) [noise=true , address=45.91.116.8:4045 , piece_id=6LRPPHPHVRYUFMIIUYTTO2CL5WRRMSWJASMJ32ROTJZIBVGMZHLA, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776240902 (509h59m8.129026s) [noise=true , address=145.79.227.7:4295 , piece_id=QYQFLNNDK5GOOLFLMLBXZEPUZLMPRYLW2UJEVBT6NLLVYCYGTPVQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.776481944 (509h59m8.129267042s) [noise=true , address=145.79.227.13:4802 , piece_id=7XS33F3ZU3BUUVCXUWU3OQ7AUTDF6BA63DFGMTH5ZDKF67GMAKRQ, seg_pos={0 0}, upload=448920]: Uploading piece... Sep 19 20:07:32.849041962 (509h59m8.20182708s) [upload=448920]: Next returned nil segment Sep 19 20:07:32.849056960 (509h59m8.201842059s) [upload=448920]: Waiting for error group managing segments... Sep 19 20:07:32.856512782 (509h59m8.20929789s) [noise=true , address=145.79.227.17:4971 , piece_id=5EXYCHVEJRO7SGDTQSFISPY6QAKSASC7QM3EPAYP2HMKVNR5UCMQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:32.857097799 (509h59m8.209882907s) [noise=true , address=145.79.227.7:4295 , piece_id=QYQFLNNDK5GOOLFLMLBXZEPUZLMPRYLW2UJEVBT6NLLVYCYGTPVQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:32.859056726 (509h59m8.211841844s) [noise=true , address=145.79.227.13:4802 , piece_id=7XS33F3ZU3BUUVCXUWU3OQ7AUTDF6BA63DFGMTH5ZDKF67GMAKRQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:35.782241505 (509h59m11.135026613s) [noise=true , address=77.90.31.21:4879 , piece_id=FSTO3T5B2EKFTPTEZIE53PUMUUDKOZKQAASA3HRWWQ7O5CQ6PFFA, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:37.016800765 (509h59m12.369585873s) [noise=true , address=77.90.31.16:4733 , piece_id=4FWGFGBKBXPB7HGUWJUNLZDOUSXPBPME246WBSPGRQVTR3OAUI6A, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:37.467822321 (509h59m12.820607459s) [noise=true , address=45.91.116.5:4299 , piece_id=EKF43QQITQAFGDZYMGUWBMROXDQFCUNLOO3TNTW7FXSPW6526NKQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:37.907472168 (509h59m13.260257306s) [noise=true , address=45.91.116.9:4262 , piece_id=4ONMQYLAFCX7VGE7IPXHLE6IW2SQCXAJAEFSIBNMBDFXHH7DRWOA, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:39.477854571 (509h59m14.830639699s) [noise=true , address=45.91.116.14:4190 , piece_id=RFZ43GBV5IWZ3XI3T37A7XZ5SOW75ABHP7P34UKG4DBN7VO63I2Q, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:07:42.773785732 (509h59m18.12657086s) [noise=true , address=45.91.116.4:4937 , piece_id=K6HXTZXJDPNZMRYKL65EYPJQ5LO4HUMXQXVHLRP3HKCSBOZKB2UQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: <nil> Sep 19 20:08:02.713152052 (509h59m38.065937149s) [noise=true , address=45.91.116.10:4083 , piece_id=U33CY6NZ66RKVNKQOPY6Y2S7DPGS77NH3CXDWCOOY5U2QX5NVU6A, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B, address:45.91.116.10:4083): protocol: write tcp 5.161.89.252:52360->45.91.116.10:4083: use of closed network connection; write tcp 5.161.89.252:52360->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:52360->45.91.116.10:4083: use of closed network connection Sep 19 20:08:02.713240237 (509h59m38.066025335s) [noise=true , address=45.91.116.13:4154 , piece_id=4TPETKLGS7W3TFAOSQE7XLPRVZNZD2JHUOYMHMIQKNNCYCPZ5SPQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN, address:45.91.116.13:4154): protocol: write tcp 5.161.89.252:33252->45.91.116.13:4154: use of closed network connection; write tcp 5.161.89.252:33252->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:33252->45.91.116.13:4154: use of closed network connection Sep 19 20:08:02.713300701 (509h59m38.0660858s) [noise=true , address=45.91.116.8:4045 , piece_id=6LRPPHPHVRYUFMIIUYTTO2CL5WRRMSWJASMJ32ROTJZIBVGMZHLA, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, address:45.91.116.8:4045): protocol: write tcp 5.161.89.252:51966->45.91.116.8:4045: use of closed network connection; write tcp 5.161.89.252:51966->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:51966->45.91.116.8:4045: use of closed network connection Sep 19 20:08:02.713319155 (509h59m38.066104264s) [noise=true , address=45.91.116.6:4973 , piece_id=5LNGCULVMYBTP7XI3MR2R5DRHP5SUEPHMA5Y7Q5UHQ2FKH4IOF4Q, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: read tcp 5.161.89.252:16098->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 19 20:08:02.713455451 (509h59m38.066240559s) [noise=true , address=45.91.116.5:4298 , piece_id=YBFPQ7DLTREC7REDG7R2UB6DWELMLEZNZFOWSD5YARJAGWGDSQNQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: write tcp 5.161.89.252:61614->45.91.116.5:4298: use of closed network connection; write tcp 5.161.89.252:61614->45.91.116.5:4298: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:61614->45.91.116.5:4298: use of closed network connection Sep 19 20:08:02.713521586 (509h59m38.066306694s) [noise=true , address=77.90.31.13:4628 , piece_id=5RPRXMWNHEJUHUAOJAO3UQIC7HUDBLLPEYBLTPYP7CBM4AAMETPQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a, address:77.90.31.13:4628): protocol: write tcp 5.161.89.252:23444->77.90.31.13:4628: use of closed network connection; write tcp 5.161.89.252:23444->77.90.31.13:4628: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:23444->77.90.31.13:4628: use of closed network connection Sep 19 20:08:02.713573042 (509h59m38.06635815s) [noise=true , address=77.90.31.15:4697 , piece_id=CIZ4AWIN3FCTWJSXTLQ6LGAQQ3T5PB7RM7UHEUKVDMLHMZDEXCHQ, seg_pos={0 0}, upload=448920]: Done uploading piece. err: ecclient: upload failed (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5, address:77.90.31.15:4697): protocol: write tcp 5.161.89.252:33274->77.90.31.15:4697: use of closed network connection; write tcp 5.161.89.252:33274->77.90.31.15:4697: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:33274->77.90.31.15:4697: use of closed network connection Sep 19 20:08:02.713591537 (509h59m38.066376645s) [seg_pos={0 0}, upload=448920]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Sep 19 20:08:02.735261062 (509h59m38.08804617s) [upload=448920]: Done uploading segments... Sep 19 20:08:02.735275329 (509h59m38.088060427s) [upload=448920]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.023348962s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.052057711s FAIL (2025-09-19 15:18:54 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.05208295s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 1.648856829s FAIL (2025-09-19 15:18:32 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.648874972s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 1.775232022s FAIL (2025-09-19 15:17:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.775255887s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.784895175s FAIL (2025-09-19 15:16:26 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.784919581s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 1.725239906s FAIL (2025-09-19 15:15:39 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.725330366s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 1.951794533s FAIL (2025-09-19 15:15:10 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 1.951821905s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.011302525s FAIL (2025-09-19 15:14:41 UTC) orbiter warning: operation: sync2.Copy uplink: encryption: context deadline exceeded Duration: 2.011324497s core uplink us1

1
...
11169
11170
11171
11172
11173
...
12704