Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [................X.............X.............................] duration:
5.317769356s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:37:51 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.317792678s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [................X...........................................] duration:
500.716934ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:37:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 500.74142ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
2.280210397s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:37:34 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.280236425s
core
uplink
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................X....X...................................] duration:
387.075574ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:37:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 387.125688ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
3 out of 60 is failed (threshold=3) [X.XX........................................................] duration:
2.220658314s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:37:22 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.220679363s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................X.............X............................] duration:
5.270251242s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:37:17 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.27028274s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................X..........................................] duration:
479.054909ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:37:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 479.075727ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.583761978s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:36:57 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.583776785s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................X.............X...........................] duration:
5.523037223s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:36:45 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.523056268s
core
uplink
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................X....X..................................] duration:
369.468273ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:36:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 369.495014ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................X.........................................] duration:
475.512713ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:36:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 475.539734ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
3.116053502s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:36:42 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.116068401s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
4 out of 60 is failed (threshold=3) [XX.XX.......................................................] duration:
2.756972945s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:36:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.756990046s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [...................X.............X..........................] duration:
4.538210986s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:36:16 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.538234309s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
2.189581586s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:35:55 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.189599408s
core
uplink
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................X....X.................................] duration:
334.119432ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:35:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 334.138889ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................X........................................] duration:
479.501864ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:35:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 479.53144ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
5 out of 60 is failed (threshold=3) [XXX.XX......................................................] duration:
2.653126892s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:35:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.653140707s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................X.............X.........................] duration:
4.536244854s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:35:09 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.536267146s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.33358181s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:34:59 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.333604041s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................X.......................................] duration:
454.367634ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:34:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 454.388905ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....................X.............X........................] duration:
4.638517049s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:34:52 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.638535434s
core
uplink
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................X....X................................] duration:
340.604494ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:34:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 340.621586ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
6 out of 60 is failed (threshold=3) [XXXX.XX.....................................................] duration:
2.633817784s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:34:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.633838543s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................X.............X.......................] duration:
5.259634025s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:34:35 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.259652931s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
7 out of 60 is failed (threshold=3) [XXXXX.XX....................................................] duration:
2.710755372s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:34:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.710777312s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................X......................................] duration:
475.438064ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:34:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 475.452502ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................X....X...............................] duration:
378.469636ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:34:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 378.486678ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
2.619506951s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:34:05 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.619532578s
core
uplink
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................X....X..............................] duration:
365.0707ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:33:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 365.091499ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
2.44821605s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:33:34 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.44823733s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................X.............X......................] duration:
5.158662224s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:33:29 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.158693692s
core
uplink
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................X....X.............................] duration:
412.909932ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:33:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 412.927195ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
2.294264561s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:33:18 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.294284749s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
8 out of 60 is failed (threshold=3) [XXXXXX.XX...................................................] duration:
2.211863136s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:33:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.211885609s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................X.....................................] duration:
498.955433ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:33:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 499.006338ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
9 out of 60 is failed (threshold=3) [XXXXXXX.XX..................................................] duration:
2.614438193s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:32:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.614458872s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [........................X.............X.....................] duration:
4.481810929s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:32:43 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.48184891s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
10 out of 60 is failed (threshold=3) [XXXXXXXX.XX.................................................] duration:
2.696662346s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:32:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.696718732s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................X....X............................] duration:
401.035629ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:32:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 401.054566ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
3.838299541s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:32:25 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.838323776s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................X....................................] duration:
506.948955ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:32:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 506.971878ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........................X.............X....................] duration:
5.176792428s trace:
https://jaeger.dev.storj.io/trace/508ccd1bd1c790cc
FAIL (2026-01-28 13:31:42 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.176804762s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
11 out of 60 is failed (threshold=3) [XXXXXXXXX.XX................................................] duration:
2.646443028s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:31:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.646480489s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
2.438423237s trace:
https://jaeger.dev.storj.io/trace/2602f4ec6aa4253
FAIL (2026-01-28 13:31:28 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.438444317s
core
uplink
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................X...................................] duration:
444.281272ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:31:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 444.297922ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................X....X...........................] duration:
320.356709ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:31:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 320.369604ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................X..................................] duration:
476.161594ms trace:
https://jaeger.dev.storj.io/trace/2a302f1903a04aa4
FAIL (2026-01-28 13:30:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 476.184778ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
12 out of 60 is failed (threshold=3) [XXXXXXXXXX.XX...............................................] duration:
2.217318707s trace:
https://jaeger.dev.storj.io/trace/7dbaa9b49c8e04d
FAIL (2026-01-28 13:30:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
load=271346]: Uploading piece...
Jan 28 12:51:50.184409631 (308h55m28.366854632s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184678012 (308h55m28.367123013s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.184816551 (308h55m28.367261552s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185220726 (308h55m28.367665726s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185238389 (308h55m28.367683389s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.185767077 (308h55m28.368212076s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186257063 (308h55m28.368702062s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.186750323 (308h55m28.369195324s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.187393415 (308h55m28.369838415s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Uploading piece...
Jan 28 12:51:50.298183821 (308h55m28.480628822s) [noise=true , address=185.167.204.205:4017 , piece_id=BEU3G2LERHFAI7LZMHRE6QE2FXHAXPQRDAEY7VLELIS2NTOOFRLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: write tcp 5.161.89.252:24554->185.167.204.205:4017: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.302365945 (308h55m28.484810935s) [noise=true , address=185.167.204.205:4008 , piece_id=HUJPT4F22AV6LXL7G4E5SS45QHO4Y63AKLELUYLJPST4E4H3UAWQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, address:185.167.204.205:4008): protocol: write tcp 5.161.89.252:24764->185.167.204.205:4008: write: connection refused; conn closed: op; piecestore: piecestore close: manager closed: conn closed: op
Jan 28 12:51:50.308007363 (308h55m28.490452363s) [noise=true , address=185.167.204.205:4020 , piece_id=E5R424DW66O6SEI2TGMZ4OA45QZHEGY4W3TYCXIF67NLJKWSKL7Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12hUxMR4vpUXuoAToGJcWvzaXsU4BcV5BFicueEaYBgyVLPQDoU, address:185.167.204.205:4020): protocol: write tcp 5.161.89.252:56272->185.167.204.205:4020: write: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:51:50.649323054 (308h55m28.831768055s) [upload=271346]: Next returned nil segment
Jan 28 12:51:50.649338894 (308h55m28.831783904s) [upload=271346]: Waiting for error group managing segments...
Jan 28 12:51:50.843663241 (308h55m29.02610824s) [noise=true , address=185.167.204.206:4057 , piece_id=TPMTJVSONKQCBHPARD34BWSBY6G2ESJX35DIT6KIE3KPHBGNFJIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.860728972 (308h55m29.043173951s) [noise=true , address=185.167.204.205:4035 , piece_id=YME24UO22KJTZ5U2ZNCDXKSIFL3MQPANUCNUE5SIIWDQSPQVVGNQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.865848004 (308h55m29.048292984s) [noise=true , address=89.149.201.147:4017 , piece_id=IZBI3TSLULG7Z7FQCL5PO4OD7BNAAYBR4N5NCFXNFOF5VLM32BRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.868016204 (308h55m29.050461174s) [noise=true , address=185.167.204.206:4071 , piece_id=7IH4EISSD763LQWLPJWJAWNBBD7576TW2KYRJUYN2LJ2JHMYROOQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.874433082 (308h55m29.056878081s) [noise=true , address=81.171.14.70:4046 , piece_id=NBJH47N24FLXZ4PW7P4ZFI3YSW6ZWI34S5ZIJMDBWGUVIMHK3LKA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876146613 (308h55m29.058591612s) [noise=true , address=81.171.14.70:4044 , piece_id=GYJNV7K3QKBRKQX5OLAVY2WN6HNZMPH63PDC2S4MRITWHXNLJDJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876299508 (308h55m29.058744509s) [noise=true , address=185.167.204.206:4044 , piece_id=WWIS2BOYXTEZXCUUE5M356A25LRZH33TEV2SIUCKRM2M6A3LZ2QA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.876910951 (308h55m29.059355951s) [noise=true , address=185.167.204.206:4058 , piece_id=PPNS6OSYK32URJXJBFVTWH6LPMHJUTIFES6G23XOOWU7LBIC7FIQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.878549080 (308h55m29.060994081s) [noise=true , address=81.171.14.70:4026 , piece_id=5FNKVITLJVTCWBT5UT3WBAXIKQRRZAGE3UKN4VAWC5XRXTP42ZRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.879597960 (308h55m29.06204296s) [noise=true , address=185.167.204.205:4034 , piece_id=MW6PCDOGOPUN7BBECKQD4RL74OFBWQJPMTDMOEC7CAGB73XITKKQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.880052580 (308h55m29.06249758s) [noise=true , address=185.167.204.205:4024 , piece_id=LBCC6OLXLDO3ZVU7OTXD4UFT6FQOHYB6DMHFPNXJW5TT4LQ4DTLQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.881826874 (308h55m29.064271874s) [noise=true , address=185.167.204.206:4072 , piece_id=GN3EORXBPUGH354PYGXTIWIZV4CMASAZNME2SMGKGE4GQTOUNHLA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.882524889 (308h55m29.064969888s) [noise=true , address=185.167.204.205:4023 , piece_id=LBS2USJN5CVWIHPOOKDYAVXHNDMPOXSUS6RMRFYADKTNKSNOW3DQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.883214717 (308h55m29.065659717s) [noise=true , address=185.167.204.206:4038 , piece_id=XHZCDD5PKKVVQCIVPYRWPBNXV2WRGFVISHMAMOBGQQBGIE76WKAA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.888303984 (308h55m29.070748984s) [noise=true , address=185.167.204.206:4043 , piece_id=HPRFDI3HA7WYH4PXNUH6HAFZTJZU3PZGWWQFTSTBTWXJKLNAZVMA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.899747231 (308h55m29.082192232s) [noise=true , address=185.167.204.205:4003 , piece_id=UUAU3ADTVJPMTBAZLBRTMWA4B7EG2ZK3BZLVWAVFS6Z3C6SOWVJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.906286668 (308h55m29.088731668s) [noise=true , address=89.149.201.147:4021 , piece_id=U7DQO73JQW2UUMREKJNLWTEBVNYOVF5VAIFZF3QK22FUGRLKKKGQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.907099736 (308h55m29.089544736s) [noise=true , address=81.171.14.70:4033 , piece_id=GWTT7ER762EM2VM4OAMLJKTCBVAAIXNISKHZZ5W23YGNWFN75MZA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.908491366 (308h55m29.090936366s) [noise=true , address=89.149.201.147:4023 , piece_id=JGYEFMC2CGAYTHYCJHHDQ4CDIVKOY56IZFHOKWNVFWV2KEBSL7TQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.913455309 (308h55m29.095900308s) [noise=true , address=81.171.14.70:4037 , piece_id=ZD2YZFWQAGDWRQYU27QF7K6NWZXJZFSM5N7RHEU5HFGJXQZWMDRA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:50.922826225 (308h55m29.105271226s) [noise=true , address=81.171.14.70:4043 , piece_id=BPIT5OOFKSEAHTXVKI334YKTJY7EN25QI3X4WZIOXJJWZP56F3SQ, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.362248849 (308h55m29.544693849s) [noise=true , address=89.149.201.147:4022 , piece_id=OSZMX3RF7BF7R4BRVR7UFRTNYHPFG5WAW5YBTYZZVPR5AYD5G2YA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:51:51.380533927 (308h55m29.562978937s) [noise=true , address=89.149.201.147:4006 , piece_id=YYH4E3SCRBGH7VVNXKO6FGTFEYNHYBXMWM43XF4ATS6GNQ6D3N4Q, seg_pos={0 0}, upload=271346]: Done uploading piece. err: <nil>
Jan 28 12:52:49.756393715 (308h56m27.938838695s) [noise=true , address=185.167.204.205:4010 , piece_id=6JGYL7KZGHFJDBYHX6UOSTQRTP63MTUQNRD3I36DHMONKRVTDPJA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:12oTu1GaRsH7fHqxLkQ9c9ikjG4zRN3NChssa5gfpLwrcEehoyV, address:185.167.204.205:4010): protocol: read tcp 5.161.89.252:30290->185.167.204.205:4010: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756442386 (308h56m27.938887355s) [noise=true , address=185.167.204.205:4012 , piece_id=WUGWJ2OZUHC5KBHFJNTBOOFLUOSSO57HG3BOQJ4HRALEELLFGUQA, seg_pos={0 0}, upload=271346]: Done uploading piece. err: ecclient: upload failed (node:1V4S6bUds1LgcrVccy5NkBJMAG3QaGFJqhsYtVboesSdH2PUFh, address:185.167.204.205:4012): protocol: read tcp 5.161.89.252:15526->185.167.204.205:4012: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 28 12:52:49.756461832 (308h56m27.938906812s) [seg_pos={0 0}, upload=271346]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Jan 28 12:52:49.861788280 (308h56m28.04423325s) [upload=271346]: Done uploading segments...
Jan 28 12:52:49.861817014 (308h56m28.044261994s) [upload=271346]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 2.217338994s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................X....X..........................] duration:
379.001795ms trace:
https://jaeger.dev.storj.io/trace/36cf2dc7bd134cfe
FAIL (2026-01-28 13:30:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 379.02148ms
edge
gateway
us-select-1
|