Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
362.230306ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:31:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 362.247928ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
409.808899ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:31:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 409.832815ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
388.727911ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:30:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 388.746165ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
395.113633ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:29:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 395.130013ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
380.937443ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:29:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 380.955417ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
363.518999ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:27:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 363.534698ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
380.380145ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:27:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 380.397498ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
346.572154ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:26:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 346.588795ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
439.167395ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:25:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 439.183304ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
416.87801ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:25:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 416.908166ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
385.500868ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:24:00 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 385.521176ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
333.892979ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:23:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 333.914287ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
396.888399ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:22:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 396.907393ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
342.846721ms trace:
https://jaeger.dev.storj.io/trace/325529c3e4960396
FAIL (2025-12-09 15:22:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 342.859926ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
2.146284813s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:55:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.146296185s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
2.648287262s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:55:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.648304564s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
2.657920489s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:54:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.657941108s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
2.624252318s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:54:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.624264701s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
2.09998501s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:53:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.100002921s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
2.240230058s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:52:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.240247069s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
2.095305258s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:51:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.095314385s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
2.166664445s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:50:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.166687037s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
2.55433327s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:50:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.554353458s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
2.552108788s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:49:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.552128927s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
2.617981082s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:48:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.618002541s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
2.649241344s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:47:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.649258505s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.63379427s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:47:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.633813697s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.580667212s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:46:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.580683794s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.175481246s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:46:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.175511011s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.167323496s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:45:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.167340789s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
2.563318832s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:45:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.563333979s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.6066313s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:44:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.606647611s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.62510665s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:44:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.625125916s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
2.646467042s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:43:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.64648697s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
2.597721074s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:42:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.597737023s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
2.578165503s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:41:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.578184589s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.625961526s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:40:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.625980761s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
2.556861357s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:39:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.556879791s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
2.57524383s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:38:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.575260502s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
2.634509167s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:37:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.634527892s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
2.195459541s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:36:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.195476794s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
2.57956647s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:35:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.57958787s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
2.267255828s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:35:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.267280085s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.617964638s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:34:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.617983503s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.134535205s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:33:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.134596832s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
2.591729263s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:32:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.591748869s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.680767553s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:31:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.680783613s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.212921071s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:30:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.212941108s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.650477653s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:29:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.65049183s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.522478339s trace:
https://jaeger.dev.storj.io/trace/5f66f153870a53ab
FAIL (2025-12-09 09:28:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 20)
======LOGS======
FQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.717808746 (89h40m11.415377051s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.718898641 (89h40m11.416466947s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.720131965 (89h40m11.417700271s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.721214246 (89h40m11.418782562s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:23.722407826 (89h40m11.419976143s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Uploading piece...
Dec 9 09:09:24.233717738 (89h40m11.931286045s) [upload=78703]: Next returned nil segment
Dec 9 09:09:24.233727325 (89h40m11.931295641s) [upload=78703]: Waiting for error group managing segments...
Dec 9 09:09:24.444659927 (89h40m12.142228243s) [noise=true , address=185.167.204.206:4043 , piece_id=CVFHS7436WDEKFRNROCKDQKITU7FWS37DAX3D3IVOY47UAPBK7DQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.474809275 (89h40m12.172377591s) [noise=true , address=185.167.204.205:4028 , piece_id=HKJKDSN24J5EDBQUQ6BHMVNN2ZEDIKLI2H2NBUS4JVPG5U2TMTCA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.486712340 (89h40m12.184280656s) [noise=true , address=185.167.204.205:4021 , piece_id=O2B2EIBNBLPBON4SO75KAGXCANZUFV7SL2FO36WMZKED6JM3GJLQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.612269996 (89h40m12.309838322s) [noise=true , address=185.167.204.205:4004 , piece_id=OWTQB5SAW7TBATFOMT4V5SJN56E73AIT3FHC6UFRVCLLF5ZML3WQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.674527530 (89h40m12.372095866s) [noise=true , address=81.171.14.70:4038 , piece_id=LAQHFVRBC2TJ6ID5TYSENWKFMP54TXCH6KQH5NQII2TWUCCHURVQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.682187378 (89h40m12.379755724s) [noise=true , address=81.171.14.70:4047 , piece_id=GWMYMODC57RJ4HKDPSRFBN4K6DBZGRHXQN26IF6KTVFFXA5QD5JQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.777428012 (89h40m12.474996349s) [noise=true , address=81.171.14.70:4034 , piece_id=2YVB527BDVKZ5NJOVC6ZLWIO3P5UAHURAJVBHB4J6GJ34GOPLQCQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.843474363 (89h40m12.541042689s) [noise=true , address=81.171.14.70:4039 , piece_id=FTQGXOKIGQYRKEGSIMRHKYWR4B7UAIYZS42CMVKVG27Y6ZR3E52A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:24.925066164 (89h40m12.622634509s) [noise=true , address=81.171.14.70:4042 , piece_id=LTNFZIUF75PHYDEWSOZCNIT3NOVYK7ZQSGUUBEFYU7VMIBC2OY7A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:25.216835034 (89h40m12.914403371s) [noise=true , address=81.171.14.70:4048 , piece_id=E26J5YTJXNYAFESFHCQYZ2Z2XHRJENROZNCNU5BKQ73JK2ODES6Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:09:26.575382517 (89h40m14.272950843s) [noise=true , address=89.149.201.147:4001 , piece_id=IMZN5QUWCUOP4DTYHYQOVDAOB3E52WEWH63KKJ4TXQHVZQURSWPA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.768648041 (89h41m3.466216378s) [noise=true , address=89.149.201.147:4007 , piece_id=XGY6FXEDPFFC7XWT46TYGATOTXRQKP4E3AZGYJSBG7YBAFT4KCOQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:15.963641791 (89h41m3.661210107s) [noise=true , address=185.167.204.206:4058 , piece_id=4RDDDWJHY7U5LPEZ465NMTOCTQNRJE3YX5WPQXN74OL5CJRMNACA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.695110269 (89h41m4.392678585s) [noise=true , address=89.149.201.147:4019 , piece_id=TELQLWRH7I7YGQJIUNSM5RY3NHT5OTTWUILTT2ES2BNB2FREAW5A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:16.774029255 (89h41m4.471597561s) [noise=true , address=185.167.204.206:4047 , piece_id=CFN7S3TY64WAMXG4S7EI3PMQ3RUAHO5QAFBSNHRH4OJ3SRQMBQSA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.645961293 (89h41m6.343529599s) [noise=true , address=185.167.204.205:4003 , piece_id=ALJRMRUYPF7DKPNWDMMPYDDPDNCCMC6COTXEVQH3COJTU5GAFKKQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:18.978217663 (89h41m6.675785968s) [noise=true , address=81.171.14.70:4033 , piece_id=7W2WDORJRGKOYOFJWSWMT5DOOPSJ5ZO2UEYKPVAJ2RHRHLJCJE3Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:19.070504941 (89h41m6.768073257s) [noise=true , address=185.167.204.205:4035 , piece_id=UZ3QMGAWGQNGI3SZRHZ6454SEWFB6GQMOJDZWV3QZXHR6JK7FY3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:20.869618151 (89h41m8.567186467s) [noise=true , address=89.149.201.147:4009 , piece_id=3FOVPTR5VRI435PMT7SJ42CPNE2UXKCRIIOXESLG4QRXGXBWDPFQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:22.930754115 (89h41m10.628322421s) [noise=true , address=185.167.204.206:4072 , piece_id=VY6QU2NGW62XCE3YACGLMJG2JECGFWHP7GCY2KRS4IGJQ2G4DLHQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: <nil>
Dec 9 09:10:23.311568748 (89h41m11.009137064s) [noise=true , address=185.167.204.206:4046 , piece_id=VFYXXDNQT5O4UH56A46ZFF2WCFQDLQ5KY3VMINITT5FSWLRWHAAQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:1YPnmxhn6aS6pLJ7VhwQkHrefetEQwH3ww3sBRyEghNvhzT8X5, address:185.167.204.206:4046): protocol: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20226->185.167.204.206:4046: use of closed network connection
Dec 9 09:10:23.311767520 (89h41m11.009335826s) [noise=true , address=81.171.14.70:4035 , piece_id=AQLYBV2DCJ4RBUFGZM7VUJ6IVNH3BQKMNF6YWYLVA6ISM7JHABFA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12eLYJKne3yP1fZ1oPDLM7Xqs3GQ3ju4ZQedAiojXdVkcxNpzeA, address:81.171.14.70:4035): protocol: read tcp 5.161.89.252:60514->81.171.14.70:4035: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.311775124 (89h41m11.009343461s) [noise=true , address=81.171.14.70:4028 , piece_id=YXF2UDTRCU5Z5GETT6PSBQYUVYDTUTOEUSXXN24IGAXEHQ6RIFUA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12s7cAtvSQB3RQj5km8v3n2gDvwTiBAB7h4cijTE951tPoytDed, address:81.171.14.70:4028): protocol: rpc: dial tcp 81.171.14.70:4028: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312081085 (89h41m11.009649391s) [noise=true , address=81.171.14.70:4027 , piece_id=G5SX2EXGEKIU5CEIZNVLYULQ2XWGT7QKWXEKEFSJTMZSTJBHFB4Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: read tcp 5.161.89.252:34412->81.171.14.70:4027: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312154953 (89h41m11.00972329s) [noise=true , address=81.171.14.70:4036 , piece_id=73FALAN3KOQU3I76QDHJF2AONMQWOFLU2T3RHZL3DJGNMYXFKYVA, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 5.161.89.252:47112->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312182976 (89h41m11.009751282s) [noise=true , address=185.167.204.206:4053 , piece_id=WY2YSKZSAWKCRXZHKSJDJEEV4DPJ3G5QVF4H2G7RMTXJFHYPR5VQ, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12SrFttJpndpv7TKJJYsKthWnGWBYQGHpS9CwqTd4mqBP9NCCm8, address:185.167.204.206:4053): protocol: read tcp 5.161.89.252:33210->185.167.204.206:4053: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312305564 (89h41m11.009873901s) [noise=true , address=185.167.204.205:4014 , piece_id=CVA4OJOYJHBDFBB72FX7LKIX6PRNWJSV377EAYAG6PRY2IQREN3A, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12ajStqnU4gCGh2ffTPhrFAP9UtB8ybC9Y4pmgnHfx5hK3rVk8q, address:185.167.204.205:4014): protocol: read tcp 5.161.89.252:42510->185.167.204.205:4014: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 9 09:10:23.312515256 (89h41m11.010083592s) [noise=true , address=185.167.204.205:4001 , piece_id=X7WATQ2NGXSJH4GSXKFJZPISRPJCFWOT6SMEAFJAA35GSYGIG34Q, seg_pos={0 0}, upload=78703]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 5.161.89.252:21910->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded
Dec 9 09:10:23.312542417 (89h41m11.010110753s) [seg_pos={0 0}, upload=78703]: Done waiting for segment. successful: 20 optimal: 26 errs: <nil>
Dec 9 09:10:23.414751899 (89h41m11.112320194s) [upload=78703]: Done uploading segments...
Dec 9 09:10:23.414775052 (89h41m11.112343348s) [upload=78703]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 20)
Duration: 2.522499147s
core
uplink
eu-select-1
eu1-placement-15
|