Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................................XX..........] duration:
369.95605ms trace:
https://jaeger.dev.storj.io/trace/246b0cf010fe0949
FAIL (2026-04-11 00:44:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 3)
======LOGS======
26m58.524717394s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.386235481 (1252h26m58.525219685s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387577275 (1252h26m58.52656147s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387788552 (1252h26m58.526772746s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388213427 (1252h26m58.527197631s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388240277 (1252h26m58.527224471s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388737148 (1252h26m58.527721343s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388783716 (1252h26m58.52776792s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389095912 (1252h26m58.528080095s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389226174 (1252h26m58.528210378s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389631124 (1252h26m58.528615308s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389748964 (1252h26m58.528733169s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390167329 (1252h26m58.529151513s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390548703 (1252h26m58.529532897s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.458961811 (1252h26m58.597945986s) [upload=1101200]: Next returned nil segment
Apr 11 00:35:39.458969326 (1252h26m58.597953499s) [upload=1101200]: Waiting for error group managing segments...
Apr 11 00:35:39.469204574 (1252h26m58.608188757s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.480291549 (1252h26m58.619275722s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.488543230 (1252h26m58.627527414s) [noise=true , address=77.90.31.11:4556 , piece_id=MK5ODY7DGS6QYT4KMH3ZO2KQYMSULWSATZSTC2ZORTBSKPLZLDTA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:54.882618410 (1252h27m14.021602593s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344483249 (1252h27m28.483467443s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3, address:145.79.227.19:4080): protocol: rpc: dial tcp 145.79.227.19:4080: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344573928 (1252h27m28.483558122s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12YcYgDWq2k7F9fwpdLAGNkrskGEUURe6QuRed7mRjMTdNHEQ8D, address:145.79.227.39:4018): protocol: rpc: dial tcp 145.79.227.39:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344717436 (1252h27m28.48370163s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:145.79.227.42:4126): protocol: rpc: dial tcp 145.79.227.42:4126: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344727846 (1252h27m28.48371204s) [noise=true , address=145.79.227.22:4510 , piece_id=PQM4DMJOIMVQXEDGXJBOEVW5ZUTVGRQHOXXOWTPHCZ7JJFHACNZQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9, address:145.79.227.22:4510): protocol: rpc: dial tcp 145.79.227.22:4510: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344829898 (1252h27m28.483814081s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344875202 (1252h27m28.483859386s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12cKhDbA4ptwux2LYNPNAJ7Qh8jECsGWd1KWaK6T6pD38bJWdt1, address:145.79.227.7:4296): protocol: rpc: dial tcp 145.79.227.7:4296: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345017278 (1252h27m28.484001493s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12SXZN7WJvcHys9BRd4Lq8NAZ3MSgHf9ZVv7SgaiJNRosdmFVpm, address:145.79.227.9:4370): protocol: rpc: dial tcp 145.79.227.9:4370: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345050681 (1252h27m28.484034895s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS, address:145.79.227.51:4412): protocol: rpc: dial tcp 145.79.227.51:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345165256 (1252h27m28.48414944s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1sKfG447ZZTQ6QJ1ohCxbfvm5Sv8u8azreBAqoC6EFbvXQdzwN, address:145.79.227.8:4335): protocol: rpc: dial tcp 145.79.227.8:4335: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345338371 (1252h27m28.484322554s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1gcSHRechDzTDwivRoAwNFfNj5Zjqvi6oJwVqzbCQinwgWtpPr, address:145.79.227.26:4658): protocol: rpc: dial tcp 145.79.227.26:4658: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345505043 (1252h27m28.484489227s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: rpc: dial tcp 145.79.227.10:4404: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345654022 (1252h27m28.484638206s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12QahNthBhqWBPgFFwAbsy5VrEuMxBkcuybmabY43sqBKKrW6ev, address:145.79.227.25:4620): protocol: rpc: dial tcp 145.79.227.25:4620: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345698466 (1252h27m28.48468265s) [seg_pos={0 0}, upload=1101200]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil>
Apr 11 00:36:09.371563390 (1252h27m28.510547573s) [upload=1101200]: Done uploading segments...
Apr 11 00:36:09.371577836 (1252h27m28.510562021s) [upload=1101200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3)
Duration: 369.972962ms
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
152.292841ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=18A526C4039F9B8C X-Request-Id=JaVZ5AqrYUp X-Xss-Protection=1; mode=block Content-Type=application/xml Server=Storj Vary=Origin Date=Sat, 11 Apr 2026 00:44:11 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/5cd83af56e68b400
FAIL (2026-04-11 00:44:11 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526C4039F9B8C</RequestId><HostId></HostId></Error>"
Duration: 152.311167ms
edge
gateway
vhost
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.................................................X..........] duration:
1.115897699s trace:
https://jaeger.dev.storj.io/trace/75735e15af9b9b94
FAIL (2026-04-11 00:44:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.115914169s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
380.779ms trace:
https://jaeger.dev.storj.io/trace/63c6acf9ae01bec7
FAIL (2026-04-11 00:43:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 380.800189ms
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
106.652073ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Vary=Origin X-Request-Id=FNoVkw39Yoq X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Amz-Request-Id=18A526BF159244C1 Date=Sat, 11 Apr 2026 00:43:50 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/4dd89e81ea31c83d
FAIL (2026-04-11 00:43:50 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526BF159244C1</RequestId><HostId></HostId></Error>"
Duration: 106.675136ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
713.477141ms trace:
https://jaeger.dev.storj.io/trace/63c6acf9ae01bec7
FAIL (2026-04-11 00:43:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 713.500235ms
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
112.483893ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Vary=Origin X-Amz-Request-Id=18A526BA9E57971C X-Request-Id=4ZhWhT6cGt3 Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Date=Sat, 11 Apr 2026 00:43:31 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/7951b532abca6241
FAIL (2026-04-11 00:43:31 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526BA9E57971C</RequestId><HostId></HostId></Error>"
Duration: 112.502739ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.25124ms remote:
"tcp:45.38.24.2:443" response headers:
Accept-Ranges=bytes Content-Length=284 Server=Storj Vary=Origin X-Amz-Request-Id=18A526BA7BB4B053 X-Request-Id=9rJuJ8DXCmV Date=Sat, 11 Apr 2026 00:43:30 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Xss-Protection=1; mode=block target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/5463114064b4320b
FAIL (2026-04-11 00:43:30 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526BA7BB4B053</RequestId><HostId></HostId></Error>"
Duration: 8.267682ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.670244359s trace:
https://jaeger.dev.storj.io/trace/23afef2367e31b52
FAIL (2026-04-11 00:43:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.670256551s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
111.231449ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml X-Amz-Request-Id=18A526B9FD12382D X-Request-Id=beeeeJJbnby X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj Vary=Origin Date=Sat, 11 Apr 2026 00:43:28 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1eb8f617c9bcffe6
FAIL (2026-04-11 00:43:28 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526B9FD12382D</RequestId><HostId></HostId></Error>"
Duration: 111.251417ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.044557591s trace:
https://jaeger.dev.storj.io/trace/763dfef2d18cd6e2
FAIL (2026-04-11 00:43:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
, upload=1101192]: Done begin upload segment.
Apr 11 00:35:00.354431566 (1252h26m19.49341576s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.355988042 (1252h26m19.494972236s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356468462 (1252h26m19.495452656s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356759707 (1252h26m19.495743891s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357047146 (1252h26m19.49603134s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357321861 (1252h26m19.496306055s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357587579 (1252h26m19.496571763s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357865299 (1252h26m19.496849493s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358122842 (1252h26m19.497107026s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358132610 (1252h26m19.497116794s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358406383 (1252h26m19.497390577s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358411694 (1252h26m19.497395877s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358709290 (1252h26m19.497693474s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359029180 (1252h26m19.498013364s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359423569 (1252h26m19.498407753s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359757234 (1252h26m19.498741418s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359839047 (1252h26m19.498823242s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.360180568 (1252h26m19.499164751s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.419188463 (1252h26m19.558172648s) [upload=1101192]: Next returned nil segment
Apr 11 00:35:00.419196418 (1252h26m19.558180602s) [upload=1101192]: Waiting for error group managing segments...
Apr 11 00:35:00.432316061 (1252h26m19.571300245s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.635711511 (1252h26m19.774695695s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.637901635 (1252h26m19.776885829s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.934800787 (1252h26m20.073784972s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.939956558 (1252h26m20.078940752s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.940039554 (1252h26m20.079023737s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941150174 (1252h26m20.080134359s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941411514 (1252h26m20.080395698s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941849996 (1252h26m20.08083418s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.960649371 (1252h26m20.099633565s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:01.307829827 (1252h26m20.446814012s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:36:00.327591793 (1252h27m19.466575967s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12bFi2opJ6FejAMwSL2qbqWoCk24QzSjT6xQYE8BytLgVJikZMT, address:145.79.227.19:4089): protocol: rpc: dial tcp 145.79.227.19:4089: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327637769 (1252h27m19.466621964s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12fNrbSfErEQRnruJCJAVvkoNwQSPWnvNtxunHn6zrQegnNAkxA, address:145.79.227.52:4461): protocol: rpc: dial tcp 145.79.227.52:4461: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327731975 (1252h27m19.46671616s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12aExoQdioFcK1qB46HjSQA6jany7ZxxgRYbBcBu75duXBEKWbi, address:145.79.227.13:4829): protocol: rpc: dial tcp 145.79.227.13:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327830250 (1252h27m19.466814424s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1JEqRG4b2YCdARm1eBnCgH4sAW4X7kRykNRZyTstpi8xavLqEg, address:145.79.227.32:4847): protocol: rpc: dial tcp 145.79.227.32:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327875465 (1252h27m19.466859659s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12KDgmvhgoRSTnByRrzSuw2N4iobi59Atdm2N2SsWZ4yoyfFHof, address:145.79.227.40:4061): protocol: rpc: dial tcp 145.79.227.40:4061: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328146733 (1252h27m19.467130927s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU, address:145.79.227.36:4542): protocol: rpc: dial tcp 145.79.227.36:4542: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328250037 (1252h27m19.467234211s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1dCvsMCm5ZRmWVqP1zr9noHVuPeyrXHjpRvdHnKoAZqqt6knx, address:145.79.227.29:4973): protocol: rpc: dial tcp 145.79.227.29:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328267800 (1252h27m19.467251985s) [seg_pos={0 0}, upload=1101192]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:36:00.349355634 (1252h27m19.488339828s) [upload=1101192]: Done uploading segments...
Apr 11 00:36:00.349374158 (1252h27m19.488358352s) [upload=1101192]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 1.044579371s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
114.290418ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=18A526B8EF9DB4EF X-Request-Id=Vxy17mvKS5V Date=Sat, 11 Apr 2026 00:43:23 GMT Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/17b27514782354fc
FAIL (2026-04-11 00:43:23 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526B8EF9DB4EF</RequestId><HostId></HostId></Error>"
Duration: 114.311117ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................................XX.........] duration:
934.505423ms trace:
https://jaeger.dev.storj.io/trace/246b0cf010fe0949
FAIL (2026-04-11 00:43:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 3)
======LOGS======
26m58.524717394s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.386235481 (1252h26m58.525219685s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387577275 (1252h26m58.52656147s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387788552 (1252h26m58.526772746s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388213427 (1252h26m58.527197631s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388240277 (1252h26m58.527224471s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388737148 (1252h26m58.527721343s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388783716 (1252h26m58.52776792s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389095912 (1252h26m58.528080095s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389226174 (1252h26m58.528210378s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389631124 (1252h26m58.528615308s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389748964 (1252h26m58.528733169s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390167329 (1252h26m58.529151513s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390548703 (1252h26m58.529532897s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.458961811 (1252h26m58.597945986s) [upload=1101200]: Next returned nil segment
Apr 11 00:35:39.458969326 (1252h26m58.597953499s) [upload=1101200]: Waiting for error group managing segments...
Apr 11 00:35:39.469204574 (1252h26m58.608188757s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.480291549 (1252h26m58.619275722s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.488543230 (1252h26m58.627527414s) [noise=true , address=77.90.31.11:4556 , piece_id=MK5ODY7DGS6QYT4KMH3ZO2KQYMSULWSATZSTC2ZORTBSKPLZLDTA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:54.882618410 (1252h27m14.021602593s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344483249 (1252h27m28.483467443s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3, address:145.79.227.19:4080): protocol: rpc: dial tcp 145.79.227.19:4080: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344573928 (1252h27m28.483558122s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12YcYgDWq2k7F9fwpdLAGNkrskGEUURe6QuRed7mRjMTdNHEQ8D, address:145.79.227.39:4018): protocol: rpc: dial tcp 145.79.227.39:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344717436 (1252h27m28.48370163s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:145.79.227.42:4126): protocol: rpc: dial tcp 145.79.227.42:4126: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344727846 (1252h27m28.48371204s) [noise=true , address=145.79.227.22:4510 , piece_id=PQM4DMJOIMVQXEDGXJBOEVW5ZUTVGRQHOXXOWTPHCZ7JJFHACNZQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9, address:145.79.227.22:4510): protocol: rpc: dial tcp 145.79.227.22:4510: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344829898 (1252h27m28.483814081s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344875202 (1252h27m28.483859386s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12cKhDbA4ptwux2LYNPNAJ7Qh8jECsGWd1KWaK6T6pD38bJWdt1, address:145.79.227.7:4296): protocol: rpc: dial tcp 145.79.227.7:4296: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345017278 (1252h27m28.484001493s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12SXZN7WJvcHys9BRd4Lq8NAZ3MSgHf9ZVv7SgaiJNRosdmFVpm, address:145.79.227.9:4370): protocol: rpc: dial tcp 145.79.227.9:4370: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345050681 (1252h27m28.484034895s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS, address:145.79.227.51:4412): protocol: rpc: dial tcp 145.79.227.51:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345165256 (1252h27m28.48414944s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1sKfG447ZZTQ6QJ1ohCxbfvm5Sv8u8azreBAqoC6EFbvXQdzwN, address:145.79.227.8:4335): protocol: rpc: dial tcp 145.79.227.8:4335: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345338371 (1252h27m28.484322554s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1gcSHRechDzTDwivRoAwNFfNj5Zjqvi6oJwVqzbCQinwgWtpPr, address:145.79.227.26:4658): protocol: rpc: dial tcp 145.79.227.26:4658: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345505043 (1252h27m28.484489227s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: rpc: dial tcp 145.79.227.10:4404: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345654022 (1252h27m28.484638206s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12QahNthBhqWBPgFFwAbsy5VrEuMxBkcuybmabY43sqBKKrW6ev, address:145.79.227.25:4620): protocol: rpc: dial tcp 145.79.227.25:4620: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345698466 (1252h27m28.48468265s) [seg_pos={0 0}, upload=1101200]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil>
Apr 11 00:36:09.371563390 (1252h27m28.510547573s) [upload=1101200]: Done uploading segments...
Apr 11 00:36:09.371577836 (1252h27m28.510562021s) [upload=1101200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3)
Duration: 934.526091ms
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
14.027324ms remote:
"tcp:45.38.24.2:443" response headers:
X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=18A526B65C388145 X-Request-Id=dGD4FSsUdhw Date=Sat, 11 Apr 2026 00:43:12 GMT Content-Type=application/xml Server=Storj Vary=Origin target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/574bc41eb3dffc43
FAIL (2026-04-11 00:43:12 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526B65C388145</RequestId><HostId></HostId></Error>"
Duration: 14.042272ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..................................................X.........] duration:
1.127827825s trace:
https://jaeger.dev.storj.io/trace/75735e15af9b9b94
FAIL (2026-04-11 00:43:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.127851429s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.525898682s trace:
https://jaeger.dev.storj.io/trace/763dfef2d18cd6e2
FAIL (2026-04-11 00:43:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
, upload=1101192]: Done begin upload segment.
Apr 11 00:35:00.354431566 (1252h26m19.49341576s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.355988042 (1252h26m19.494972236s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356468462 (1252h26m19.495452656s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356759707 (1252h26m19.495743891s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357047146 (1252h26m19.49603134s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357321861 (1252h26m19.496306055s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357587579 (1252h26m19.496571763s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357865299 (1252h26m19.496849493s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358122842 (1252h26m19.497107026s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358132610 (1252h26m19.497116794s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358406383 (1252h26m19.497390577s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358411694 (1252h26m19.497395877s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358709290 (1252h26m19.497693474s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359029180 (1252h26m19.498013364s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359423569 (1252h26m19.498407753s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359757234 (1252h26m19.498741418s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359839047 (1252h26m19.498823242s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.360180568 (1252h26m19.499164751s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.419188463 (1252h26m19.558172648s) [upload=1101192]: Next returned nil segment
Apr 11 00:35:00.419196418 (1252h26m19.558180602s) [upload=1101192]: Waiting for error group managing segments...
Apr 11 00:35:00.432316061 (1252h26m19.571300245s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.635711511 (1252h26m19.774695695s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.637901635 (1252h26m19.776885829s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.934800787 (1252h26m20.073784972s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.939956558 (1252h26m20.078940752s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.940039554 (1252h26m20.079023737s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941150174 (1252h26m20.080134359s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941411514 (1252h26m20.080395698s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941849996 (1252h26m20.08083418s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.960649371 (1252h26m20.099633565s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:01.307829827 (1252h26m20.446814012s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:36:00.327591793 (1252h27m19.466575967s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12bFi2opJ6FejAMwSL2qbqWoCk24QzSjT6xQYE8BytLgVJikZMT, address:145.79.227.19:4089): protocol: rpc: dial tcp 145.79.227.19:4089: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327637769 (1252h27m19.466621964s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12fNrbSfErEQRnruJCJAVvkoNwQSPWnvNtxunHn6zrQegnNAkxA, address:145.79.227.52:4461): protocol: rpc: dial tcp 145.79.227.52:4461: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327731975 (1252h27m19.46671616s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12aExoQdioFcK1qB46HjSQA6jany7ZxxgRYbBcBu75duXBEKWbi, address:145.79.227.13:4829): protocol: rpc: dial tcp 145.79.227.13:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327830250 (1252h27m19.466814424s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1JEqRG4b2YCdARm1eBnCgH4sAW4X7kRykNRZyTstpi8xavLqEg, address:145.79.227.32:4847): protocol: rpc: dial tcp 145.79.227.32:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327875465 (1252h27m19.466859659s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12KDgmvhgoRSTnByRrzSuw2N4iobi59Atdm2N2SsWZ4yoyfFHof, address:145.79.227.40:4061): protocol: rpc: dial tcp 145.79.227.40:4061: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328146733 (1252h27m19.467130927s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU, address:145.79.227.36:4542): protocol: rpc: dial tcp 145.79.227.36:4542: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328250037 (1252h27m19.467234211s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1dCvsMCm5ZRmWVqP1zr9noHVuPeyrXHjpRvdHnKoAZqqt6knx, address:145.79.227.29:4973): protocol: rpc: dial tcp 145.79.227.29:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328267800 (1252h27m19.467251985s) [seg_pos={0 0}, upload=1101192]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:36:00.349355634 (1252h27m19.488339828s) [upload=1101192]: Done uploading segments...
Apr 11 00:36:00.349374158 (1252h27m19.488358352s) [upload=1101192]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 1.525919301s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................................................XX........] duration:
411.646354ms trace:
https://jaeger.dev.storj.io/trace/246b0cf010fe0949
FAIL (2026-04-11 00:42:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 3)
======LOGS======
26m58.524717394s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.386235481 (1252h26m58.525219685s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387577275 (1252h26m58.52656147s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387788552 (1252h26m58.526772746s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388213427 (1252h26m58.527197631s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388240277 (1252h26m58.527224471s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388737148 (1252h26m58.527721343s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388783716 (1252h26m58.52776792s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389095912 (1252h26m58.528080095s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389226174 (1252h26m58.528210378s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389631124 (1252h26m58.528615308s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389748964 (1252h26m58.528733169s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390167329 (1252h26m58.529151513s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390548703 (1252h26m58.529532897s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.458961811 (1252h26m58.597945986s) [upload=1101200]: Next returned nil segment
Apr 11 00:35:39.458969326 (1252h26m58.597953499s) [upload=1101200]: Waiting for error group managing segments...
Apr 11 00:35:39.469204574 (1252h26m58.608188757s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.480291549 (1252h26m58.619275722s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.488543230 (1252h26m58.627527414s) [noise=true , address=77.90.31.11:4556 , piece_id=MK5ODY7DGS6QYT4KMH3ZO2KQYMSULWSATZSTC2ZORTBSKPLZLDTA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:54.882618410 (1252h27m14.021602593s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344483249 (1252h27m28.483467443s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3, address:145.79.227.19:4080): protocol: rpc: dial tcp 145.79.227.19:4080: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344573928 (1252h27m28.483558122s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12YcYgDWq2k7F9fwpdLAGNkrskGEUURe6QuRed7mRjMTdNHEQ8D, address:145.79.227.39:4018): protocol: rpc: dial tcp 145.79.227.39:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344717436 (1252h27m28.48370163s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:145.79.227.42:4126): protocol: rpc: dial tcp 145.79.227.42:4126: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344727846 (1252h27m28.48371204s) [noise=true , address=145.79.227.22:4510 , piece_id=PQM4DMJOIMVQXEDGXJBOEVW5ZUTVGRQHOXXOWTPHCZ7JJFHACNZQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9, address:145.79.227.22:4510): protocol: rpc: dial tcp 145.79.227.22:4510: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344829898 (1252h27m28.483814081s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344875202 (1252h27m28.483859386s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12cKhDbA4ptwux2LYNPNAJ7Qh8jECsGWd1KWaK6T6pD38bJWdt1, address:145.79.227.7:4296): protocol: rpc: dial tcp 145.79.227.7:4296: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345017278 (1252h27m28.484001493s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12SXZN7WJvcHys9BRd4Lq8NAZ3MSgHf9ZVv7SgaiJNRosdmFVpm, address:145.79.227.9:4370): protocol: rpc: dial tcp 145.79.227.9:4370: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345050681 (1252h27m28.484034895s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS, address:145.79.227.51:4412): protocol: rpc: dial tcp 145.79.227.51:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345165256 (1252h27m28.48414944s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1sKfG447ZZTQ6QJ1ohCxbfvm5Sv8u8azreBAqoC6EFbvXQdzwN, address:145.79.227.8:4335): protocol: rpc: dial tcp 145.79.227.8:4335: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345338371 (1252h27m28.484322554s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1gcSHRechDzTDwivRoAwNFfNj5Zjqvi6oJwVqzbCQinwgWtpPr, address:145.79.227.26:4658): protocol: rpc: dial tcp 145.79.227.26:4658: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345505043 (1252h27m28.484489227s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: rpc: dial tcp 145.79.227.10:4404: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345654022 (1252h27m28.484638206s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12QahNthBhqWBPgFFwAbsy5VrEuMxBkcuybmabY43sqBKKrW6ev, address:145.79.227.25:4620): protocol: rpc: dial tcp 145.79.227.25:4620: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345698466 (1252h27m28.48468265s) [seg_pos={0 0}, upload=1101200]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil>
Apr 11 00:36:09.371563390 (1252h27m28.510547573s) [upload=1101200]: Done uploading segments...
Apr 11 00:36:09.371577836 (1252h27m28.510562021s) [upload=1101200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3)
Duration: 411.68624ms
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
108.065976ms remote:
"tcp:136.0.77.3:443" response headers:
Date=Sat, 11 Apr 2026 00:42:40 GMT Accept-Ranges=bytes Content-Length=274 Server=Storj X-Request-Id=MfkPePiJizV X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18A526AEBF5B5D26 target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/601f037c8144c5c1
FAIL (2026-04-11 00:42:40 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526AEBF5B5D26</RequestId><HostId></HostId></Error>"
Duration: 108.082596ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
113.220223ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=18A526ACAC6E3B62 X-Request-Id=ZppqFvPbqkL X-Xss-Protection=1; mode=block Date=Sat, 11 Apr 2026 00:42:31 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/3bbca12323c18187
FAIL (2026-04-11 00:42:31 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526ACAC6E3B62</RequestId><HostId></HostId></Error>"
Duration: 113.238358ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...................................................X........] duration:
1.09043931s trace:
https://jaeger.dev.storj.io/trace/75735e15af9b9b94
FAIL (2026-04-11 00:42:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.090463155s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
378.955964ms trace:
https://jaeger.dev.storj.io/trace/63c6acf9ae01bec7
FAIL (2026-04-11 00:42:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 378.978897ms
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.662237154s trace:
https://jaeger.dev.storj.io/trace/23afef2367e31b52
FAIL (2026-04-11 00:42:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.662259967s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.544956857s trace:
https://jaeger.dev.storj.io/trace/763dfef2d18cd6e2
FAIL (2026-04-11 00:42:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
, upload=1101192]: Done begin upload segment.
Apr 11 00:35:00.354431566 (1252h26m19.49341576s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.355988042 (1252h26m19.494972236s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356468462 (1252h26m19.495452656s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356759707 (1252h26m19.495743891s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357047146 (1252h26m19.49603134s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357321861 (1252h26m19.496306055s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357587579 (1252h26m19.496571763s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357865299 (1252h26m19.496849493s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358122842 (1252h26m19.497107026s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358132610 (1252h26m19.497116794s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358406383 (1252h26m19.497390577s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358411694 (1252h26m19.497395877s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358709290 (1252h26m19.497693474s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359029180 (1252h26m19.498013364s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359423569 (1252h26m19.498407753s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359757234 (1252h26m19.498741418s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359839047 (1252h26m19.498823242s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.360180568 (1252h26m19.499164751s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.419188463 (1252h26m19.558172648s) [upload=1101192]: Next returned nil segment
Apr 11 00:35:00.419196418 (1252h26m19.558180602s) [upload=1101192]: Waiting for error group managing segments...
Apr 11 00:35:00.432316061 (1252h26m19.571300245s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.635711511 (1252h26m19.774695695s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.637901635 (1252h26m19.776885829s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.934800787 (1252h26m20.073784972s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.939956558 (1252h26m20.078940752s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.940039554 (1252h26m20.079023737s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941150174 (1252h26m20.080134359s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941411514 (1252h26m20.080395698s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941849996 (1252h26m20.08083418s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.960649371 (1252h26m20.099633565s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:01.307829827 (1252h26m20.446814012s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:36:00.327591793 (1252h27m19.466575967s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12bFi2opJ6FejAMwSL2qbqWoCk24QzSjT6xQYE8BytLgVJikZMT, address:145.79.227.19:4089): protocol: rpc: dial tcp 145.79.227.19:4089: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327637769 (1252h27m19.466621964s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12fNrbSfErEQRnruJCJAVvkoNwQSPWnvNtxunHn6zrQegnNAkxA, address:145.79.227.52:4461): protocol: rpc: dial tcp 145.79.227.52:4461: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327731975 (1252h27m19.46671616s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12aExoQdioFcK1qB46HjSQA6jany7ZxxgRYbBcBu75duXBEKWbi, address:145.79.227.13:4829): protocol: rpc: dial tcp 145.79.227.13:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327830250 (1252h27m19.466814424s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1JEqRG4b2YCdARm1eBnCgH4sAW4X7kRykNRZyTstpi8xavLqEg, address:145.79.227.32:4847): protocol: rpc: dial tcp 145.79.227.32:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327875465 (1252h27m19.466859659s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12KDgmvhgoRSTnByRrzSuw2N4iobi59Atdm2N2SsWZ4yoyfFHof, address:145.79.227.40:4061): protocol: rpc: dial tcp 145.79.227.40:4061: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328146733 (1252h27m19.467130927s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU, address:145.79.227.36:4542): protocol: rpc: dial tcp 145.79.227.36:4542: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328250037 (1252h27m19.467234211s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1dCvsMCm5ZRmWVqP1zr9noHVuPeyrXHjpRvdHnKoAZqqt6knx, address:145.79.227.29:4973): protocol: rpc: dial tcp 145.79.227.29:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328267800 (1252h27m19.467251985s) [seg_pos={0 0}, upload=1101192]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:36:00.349355634 (1252h27m19.488339828s) [upload=1101192]: Done uploading segments...
Apr 11 00:36:00.349374158 (1252h27m19.488358352s) [upload=1101192]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 1.544973929s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................................XX.......] duration:
351.103557ms trace:
https://jaeger.dev.storj.io/trace/246b0cf010fe0949
FAIL (2026-04-11 00:42:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 3)
======LOGS======
26m58.524717394s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.386235481 (1252h26m58.525219685s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387577275 (1252h26m58.52656147s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387788552 (1252h26m58.526772746s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388213427 (1252h26m58.527197631s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388240277 (1252h26m58.527224471s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388737148 (1252h26m58.527721343s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388783716 (1252h26m58.52776792s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389095912 (1252h26m58.528080095s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389226174 (1252h26m58.528210378s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389631124 (1252h26m58.528615308s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389748964 (1252h26m58.528733169s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390167329 (1252h26m58.529151513s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390548703 (1252h26m58.529532897s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.458961811 (1252h26m58.597945986s) [upload=1101200]: Next returned nil segment
Apr 11 00:35:39.458969326 (1252h26m58.597953499s) [upload=1101200]: Waiting for error group managing segments...
Apr 11 00:35:39.469204574 (1252h26m58.608188757s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.480291549 (1252h26m58.619275722s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.488543230 (1252h26m58.627527414s) [noise=true , address=77.90.31.11:4556 , piece_id=MK5ODY7DGS6QYT4KMH3ZO2KQYMSULWSATZSTC2ZORTBSKPLZLDTA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:54.882618410 (1252h27m14.021602593s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344483249 (1252h27m28.483467443s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3, address:145.79.227.19:4080): protocol: rpc: dial tcp 145.79.227.19:4080: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344573928 (1252h27m28.483558122s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12YcYgDWq2k7F9fwpdLAGNkrskGEUURe6QuRed7mRjMTdNHEQ8D, address:145.79.227.39:4018): protocol: rpc: dial tcp 145.79.227.39:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344717436 (1252h27m28.48370163s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:145.79.227.42:4126): protocol: rpc: dial tcp 145.79.227.42:4126: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344727846 (1252h27m28.48371204s) [noise=true , address=145.79.227.22:4510 , piece_id=PQM4DMJOIMVQXEDGXJBOEVW5ZUTVGRQHOXXOWTPHCZ7JJFHACNZQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9, address:145.79.227.22:4510): protocol: rpc: dial tcp 145.79.227.22:4510: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344829898 (1252h27m28.483814081s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344875202 (1252h27m28.483859386s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12cKhDbA4ptwux2LYNPNAJ7Qh8jECsGWd1KWaK6T6pD38bJWdt1, address:145.79.227.7:4296): protocol: rpc: dial tcp 145.79.227.7:4296: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345017278 (1252h27m28.484001493s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12SXZN7WJvcHys9BRd4Lq8NAZ3MSgHf9ZVv7SgaiJNRosdmFVpm, address:145.79.227.9:4370): protocol: rpc: dial tcp 145.79.227.9:4370: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345050681 (1252h27m28.484034895s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS, address:145.79.227.51:4412): protocol: rpc: dial tcp 145.79.227.51:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345165256 (1252h27m28.48414944s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1sKfG447ZZTQ6QJ1ohCxbfvm5Sv8u8azreBAqoC6EFbvXQdzwN, address:145.79.227.8:4335): protocol: rpc: dial tcp 145.79.227.8:4335: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345338371 (1252h27m28.484322554s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1gcSHRechDzTDwivRoAwNFfNj5Zjqvi6oJwVqzbCQinwgWtpPr, address:145.79.227.26:4658): protocol: rpc: dial tcp 145.79.227.26:4658: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345505043 (1252h27m28.484489227s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: rpc: dial tcp 145.79.227.10:4404: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345654022 (1252h27m28.484638206s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12QahNthBhqWBPgFFwAbsy5VrEuMxBkcuybmabY43sqBKKrW6ev, address:145.79.227.25:4620): protocol: rpc: dial tcp 145.79.227.25:4620: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345698466 (1252h27m28.48468265s) [seg_pos={0 0}, upload=1101200]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil>
Apr 11 00:36:09.371563390 (1252h27m28.510547573s) [upload=1101200]: Done uploading segments...
Apr 11 00:36:09.371577836 (1252h27m28.510562021s) [upload=1101200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3)
Duration: 351.1214ms
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
111.291728ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18A526A8639C3D63 X-Request-Id=Yuhgxet7jNE X-Xss-Protection=1; mode=block Date=Sat, 11 Apr 2026 00:42:12 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/69a67e02f74ed5dd
FAIL (2026-04-11 00:42:12 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526A8639C3D63</RequestId><HostId></HostId></Error>"
Duration: 111.310833ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
9.716656ms remote:
"tcp:45.38.24.2:443" response headers:
X-Amz-Request-Id=18A526A81AB39C27 X-Xss-Protection=1; mode=block Accept-Ranges=bytes Server=Storj X-Request-Id=HK8hxeF3cyr Date=Sat, 11 Apr 2026 00:42:11 GMT Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/146c09cea9e786cc
FAIL (2026-04-11 00:42:11 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526A81AB39C27</RequestId><HostId></HostId></Error>"
Duration: 9.737655ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
112.644245ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=18A526A6FC6A050B X-Request-Id=UW7568picYt X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Date=Sat, 11 Apr 2026 00:42:06 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/fc40adb8ad6d2ba
FAIL (2026-04-11 00:42:06 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526A6FC6A050B</RequestId><HostId></HostId></Error>"
Duration: 112.660455ms
edge
gateway
vhost
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
113.372255ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18A526A648E3A3C2 Date=Sat, 11 Apr 2026 00:42:03 GMT Server=Storj X-Request-Id=imy1EEA7pvx X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/159af2b4f6595373
FAIL (2026-04-11 00:42:03 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526A648E3A3C2</RequestId><HostId></HostId></Error>"
Duration: 113.389818ms
edge
gateway
vhost
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......] duration:
1.194199515s trace:
https://jaeger.dev.storj.io/trace/75735e15af9b9b94
FAIL (2026-04-11 00:42:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.19421895s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.65808ms remote:
"tcp:45.38.24.2:443" response headers:
Date=Sat, 11 Apr 2026 00:41:51 GMT Accept-Ranges=bytes Vary=Origin X-Xss-Protection=1; mode=block Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Amz-Request-Id=18A526A3659D7981 X-Request-Id=ger3XqaSPzU target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/39594d39bdb2fbf0
FAIL (2026-04-11 00:41:51 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526A3659D7981</RequestId><HostId></HostId></Error>"
Duration: 13.684129ms
edge
gateway
vhost
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
394.757472ms trace:
https://jaeger.dev.storj.io/trace/63c6acf9ae01bec7
FAIL (2026-04-11 00:41:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 394.774815ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.542540017s trace:
https://jaeger.dev.storj.io/trace/763dfef2d18cd6e2
FAIL (2026-04-11 00:41:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
, upload=1101192]: Done begin upload segment.
Apr 11 00:35:00.354431566 (1252h26m19.49341576s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.355988042 (1252h26m19.494972236s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356468462 (1252h26m19.495452656s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356759707 (1252h26m19.495743891s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357047146 (1252h26m19.49603134s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357321861 (1252h26m19.496306055s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357587579 (1252h26m19.496571763s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357865299 (1252h26m19.496849493s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358122842 (1252h26m19.497107026s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358132610 (1252h26m19.497116794s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358406383 (1252h26m19.497390577s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358411694 (1252h26m19.497395877s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358709290 (1252h26m19.497693474s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359029180 (1252h26m19.498013364s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359423569 (1252h26m19.498407753s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359757234 (1252h26m19.498741418s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359839047 (1252h26m19.498823242s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.360180568 (1252h26m19.499164751s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.419188463 (1252h26m19.558172648s) [upload=1101192]: Next returned nil segment
Apr 11 00:35:00.419196418 (1252h26m19.558180602s) [upload=1101192]: Waiting for error group managing segments...
Apr 11 00:35:00.432316061 (1252h26m19.571300245s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.635711511 (1252h26m19.774695695s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.637901635 (1252h26m19.776885829s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.934800787 (1252h26m20.073784972s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.939956558 (1252h26m20.078940752s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.940039554 (1252h26m20.079023737s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941150174 (1252h26m20.080134359s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941411514 (1252h26m20.080395698s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941849996 (1252h26m20.08083418s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.960649371 (1252h26m20.099633565s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:01.307829827 (1252h26m20.446814012s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:36:00.327591793 (1252h27m19.466575967s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12bFi2opJ6FejAMwSL2qbqWoCk24QzSjT6xQYE8BytLgVJikZMT, address:145.79.227.19:4089): protocol: rpc: dial tcp 145.79.227.19:4089: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327637769 (1252h27m19.466621964s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12fNrbSfErEQRnruJCJAVvkoNwQSPWnvNtxunHn6zrQegnNAkxA, address:145.79.227.52:4461): protocol: rpc: dial tcp 145.79.227.52:4461: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327731975 (1252h27m19.46671616s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12aExoQdioFcK1qB46HjSQA6jany7ZxxgRYbBcBu75duXBEKWbi, address:145.79.227.13:4829): protocol: rpc: dial tcp 145.79.227.13:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327830250 (1252h27m19.466814424s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1JEqRG4b2YCdARm1eBnCgH4sAW4X7kRykNRZyTstpi8xavLqEg, address:145.79.227.32:4847): protocol: rpc: dial tcp 145.79.227.32:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327875465 (1252h27m19.466859659s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12KDgmvhgoRSTnByRrzSuw2N4iobi59Atdm2N2SsWZ4yoyfFHof, address:145.79.227.40:4061): protocol: rpc: dial tcp 145.79.227.40:4061: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328146733 (1252h27m19.467130927s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU, address:145.79.227.36:4542): protocol: rpc: dial tcp 145.79.227.36:4542: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328250037 (1252h27m19.467234211s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1dCvsMCm5ZRmWVqP1zr9noHVuPeyrXHjpRvdHnKoAZqqt6knx, address:145.79.227.29:4973): protocol: rpc: dial tcp 145.79.227.29:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328267800 (1252h27m19.467251985s) [seg_pos={0 0}, upload=1101192]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:36:00.349355634 (1252h27m19.488339828s) [upload=1101192]: Done uploading segments...
Apr 11 00:36:00.349374158 (1252h27m19.488358352s) [upload=1101192]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 1.542558462s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
117.483109ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin X-Amz-Request-Id=18A5269F5156E036 Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Request-Id=A8AckjrfrTS X-Xss-Protection=1; mode=block Date=Sat, 11 Apr 2026 00:41:33 GMT Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/14a6477d4865a4e8
FAIL (2026-04-11 00:41:33 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A5269F5156E036</RequestId><HostId></HostId></Error>"
Duration: 117.50069ms
edge
gateway
vhost
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.57846536s trace:
https://jaeger.dev.storj.io/trace/23afef2367e31b52
FAIL (2026-04-11 00:41:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.578483053s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
107.411981ms remote:
"tcp:136.0.77.3:443" response headers:
X-Amz-Request-Id=18A526995C1CBD3E X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Vary=Origin X-Request-Id=NDyCKkEHAAR Date=Sat, 11 Apr 2026 00:41:08 GMT Content-Length=274 Content-Type=application/xml Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/2cf10ffdd7386a0f
FAIL (2026-04-11 00:41:08 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A526995C1CBD3E</RequestId><HostId></HostId></Error>"
Duration: 107.43233ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
1.196272426s trace:
https://jaeger.dev.storj.io/trace/75735e15af9b9b94
FAIL (2026-04-11 00:41:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.196288416s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................................XX......] duration:
538.924486ms trace:
https://jaeger.dev.storj.io/trace/246b0cf010fe0949
FAIL (2026-04-11 00:41:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 3)
======LOGS======
26m58.524717394s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.386235481 (1252h26m58.525219685s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387577275 (1252h26m58.52656147s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.387788552 (1252h26m58.526772746s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388213427 (1252h26m58.527197631s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388240277 (1252h26m58.527224471s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388737148 (1252h26m58.527721343s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.388783716 (1252h26m58.52776792s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389095912 (1252h26m58.528080095s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389226174 (1252h26m58.528210378s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389631124 (1252h26m58.528615308s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.389748964 (1252h26m58.528733169s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390167329 (1252h26m58.529151513s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.390548703 (1252h26m58.529532897s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Uploading piece...
Apr 11 00:35:39.458961811 (1252h26m58.597945986s) [upload=1101200]: Next returned nil segment
Apr 11 00:35:39.458969326 (1252h26m58.597953499s) [upload=1101200]: Waiting for error group managing segments...
Apr 11 00:35:39.469204574 (1252h26m58.608188757s) [noise=true , address=77.90.31.14:4666 , piece_id=XEXKCHNUQQRMTBP3I5EXDDEVCTD47N5PXBTNHZUH44R75XMSEH3Q, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.480291549 (1252h26m58.619275722s) [noise=true , address=77.90.31.13:4626 , piece_id=MLPK7UBNLEO7OLG367ZD6DWMFTQNXAYZ5CE73L2HBX5BUDNNPVRA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:39.488543230 (1252h26m58.627527414s) [noise=true , address=77.90.31.11:4556 , piece_id=MK5ODY7DGS6QYT4KMH3ZO2KQYMSULWSATZSTC2ZORTBSKPLZLDTA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: <nil>
Apr 11 00:35:54.882618410 (1252h27m14.021602593s) [noise=true , address=145.79.227.52:4445 , piece_id=YNXTHR6G2VSWZORO4ZHZDRZO5KDQDPGPXAQ7CTZPRG7CH7PM24PQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: connect: no route to host; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344483249 (1252h27m28.483467443s) [noise=true , address=145.79.227.19:4080 , piece_id=4EZ66JMT5KDVFLKFLLNPBON2AGJJ7V63IEJ3UC5ZCMHW72C6KT6A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12KK51wudKiZb39Ku7wZhgBYEYmgsZTHfjiNNLmpBU9uuNFapt3, address:145.79.227.19:4080): protocol: rpc: dial tcp 145.79.227.19:4080: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344573928 (1252h27m28.483558122s) [noise=true , address=145.79.227.39:4018 , piece_id=AHG5ROZ5MJNH2J4RALU647NFVND6RYSSQKOIN62IXJL7APNF3PLQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12YcYgDWq2k7F9fwpdLAGNkrskGEUURe6QuRed7mRjMTdNHEQ8D, address:145.79.227.39:4018): protocol: rpc: dial tcp 145.79.227.39:4018: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344717436 (1252h27m28.48370163s) [noise=true , address=145.79.227.42:4126 , piece_id=3SOIHC6OEKURVHGYN7HTBCT3JRZNF5WZ6H3KWSXDKL7LQDKWLHDQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12dMWFrq2NMpnMzuPAphWerntC5kkrcnzRJZBA116uzJzyvk6fH, address:145.79.227.42:4126): protocol: rpc: dial tcp 145.79.227.42:4126: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344727846 (1252h27m28.48371204s) [noise=true , address=145.79.227.22:4510 , piece_id=PQM4DMJOIMVQXEDGXJBOEVW5ZUTVGRQHOXXOWTPHCZ7JJFHACNZQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12touQ91P4kPaTdodCZ2EfbbNE84Yto2kYSPhQ6UMZTR5mtH5r9, address:145.79.227.22:4510): protocol: rpc: dial tcp 145.79.227.22:4510: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344829898 (1252h27m28.483814081s) [noise=true , address=145.79.227.11:4731 , piece_id=VDC3IRBVR34ROMWITOC7INZKFVPIY6P3SIJ4Y5T5UQDDIE2F5OIQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: rpc: dial tcp 145.79.227.11:4731: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.344875202 (1252h27m28.483859386s) [noise=true , address=145.79.227.7:4296 , piece_id=AGLZROZVCWRJBGTJXJMW4Y7HURG3WB6OEZGNCCD2BHFNVUGQQ6UQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12cKhDbA4ptwux2LYNPNAJ7Qh8jECsGWd1KWaK6T6pD38bJWdt1, address:145.79.227.7:4296): protocol: rpc: dial tcp 145.79.227.7:4296: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345017278 (1252h27m28.484001493s) [noise=true , address=145.79.227.9:4370 , piece_id=75WD2SGLBT3Y5OWO7GYIV64NOUKWZQZNIDXOV2MRRWOREOWM5KAQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12SXZN7WJvcHys9BRd4Lq8NAZ3MSgHf9ZVv7SgaiJNRosdmFVpm, address:145.79.227.9:4370): protocol: rpc: dial tcp 145.79.227.9:4370: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345050681 (1252h27m28.484034895s) [noise=true , address=145.79.227.51:4412 , piece_id=FU3QNM3JOCZSXA2YABOSSOZPXBWEKDVTZFSOYXTA52YG3GQ3PCYQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12vBvEfCYZxsv17cVsQTbCQ4YjZfiD4RXjWyz7Ka9ZZrtvzC7SS, address:145.79.227.51:4412): protocol: rpc: dial tcp 145.79.227.51:4412: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345165256 (1252h27m28.48414944s) [noise=true , address=145.79.227.8:4335 , piece_id=WH3U7VO5KYL4POVCCXQU4L2AYNQ6TXL3F2ZT3N3XPVBFP4RZ6KKA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1sKfG447ZZTQ6QJ1ohCxbfvm5Sv8u8azreBAqoC6EFbvXQdzwN, address:145.79.227.8:4335): protocol: rpc: dial tcp 145.79.227.8:4335: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345338371 (1252h27m28.484322554s) [noise=true , address=145.79.227.26:4658 , piece_id=PT5ZSW5VA6LSGWYET4PEP4OBAEQXSSYSWXKWPELWI5XYFGLOU3IA, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:1gcSHRechDzTDwivRoAwNFfNj5Zjqvi6oJwVqzbCQinwgWtpPr, address:145.79.227.26:4658): protocol: rpc: dial tcp 145.79.227.26:4658: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345505043 (1252h27m28.484489227s) [noise=true , address=145.79.227.10:4404 , piece_id=VCYHMNYKDFFGRYJNHGJLT4YIBQPS6AHTJLI77Y4AR2RGIWLWF52A, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: rpc: dial tcp 145.79.227.10:4404: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345654022 (1252h27m28.484638206s) [noise=true , address=145.79.227.25:4620 , piece_id=PQEN3BWUDSCGICRLUU4IKYHPTT7Q3CEZRYMOBMHORFYEV3B2N7XQ, seg_pos={0 0}, upload=1101200]: Done uploading piece. err: ecclient: upload failed (node:12QahNthBhqWBPgFFwAbsy5VrEuMxBkcuybmabY43sqBKKrW6ev, address:145.79.227.25:4620): protocol: rpc: dial tcp 145.79.227.25:4620: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:09.345698466 (1252h27m28.48468265s) [seg_pos={0 0}, upload=1101200]: Done waiting for segment. successful: 3 optimal: 13 errs: <nil>
Apr 11 00:36:09.371563390 (1252h27m28.510547573s) [upload=1101200]: Done uploading segments...
Apr 11 00:36:09.371577836 (1252h27m28.510562021s) [upload=1101200]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 3)
Duration: 538.943711ms
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
112.458308ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=AYSud5rbTr9 X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=274 Server=Storj X-Amz-Request-Id=18A52696F97E7FA7 Date=Sat, 11 Apr 2026 00:40:57 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/27cba23eadc5b9d0
FAIL (2026-04-11 00:40:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A52696F97E7FA7</RequestId><HostId></HostId></Error>"
Duration: 112.523048ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
112.469499ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=E2Q1wWuHmcF Date=Sat, 11 Apr 2026 00:40:57 GMT Content-Type=application/xml Vary=Origin X-Amz-Request-Id=18A52696F0B97D6D X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/7065ac534324717f
FAIL (2026-04-11 00:40:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A52696F0B97D6D</RequestId><HostId></HostId></Error>"
Duration: 112.48575ms
edge
gateway
vhost
bgp
ap1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
411.956711ms trace:
https://jaeger.dev.storj.io/trace/63c6acf9ae01bec7
FAIL (2026-04-11 00:40:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 411.979433ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.669308032s trace:
https://jaeger.dev.storj.io/trace/763dfef2d18cd6e2
FAIL (2026-04-11 00:40:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 14 but got 11)
======LOGS======
, upload=1101192]: Done begin upload segment.
Apr 11 00:35:00.354431566 (1252h26m19.49341576s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.355988042 (1252h26m19.494972236s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356468462 (1252h26m19.495452656s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.356759707 (1252h26m19.495743891s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357047146 (1252h26m19.49603134s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357321861 (1252h26m19.496306055s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357587579 (1252h26m19.496571763s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.357865299 (1252h26m19.496849493s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358122842 (1252h26m19.497107026s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358132610 (1252h26m19.497116794s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358406383 (1252h26m19.497390577s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358411694 (1252h26m19.497395877s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.358709290 (1252h26m19.497693474s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359029180 (1252h26m19.498013364s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359423569 (1252h26m19.498407753s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359757234 (1252h26m19.498741418s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.359839047 (1252h26m19.498823242s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.360180568 (1252h26m19.499164751s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Uploading piece...
Apr 11 00:35:00.419188463 (1252h26m19.558172648s) [upload=1101192]: Next returned nil segment
Apr 11 00:35:00.419196418 (1252h26m19.558180602s) [upload=1101192]: Waiting for error group managing segments...
Apr 11 00:35:00.432316061 (1252h26m19.571300245s) [noise=true , address=45.91.116.11:4135 , piece_id=OLKIKJFU2V7624ARI2VOEWWJX7ECEMPCJIWUWEBOQWYSJRHTLXDA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.635711511 (1252h26m19.774695695s) [noise=true , address=167.17.72.69:4710 , piece_id=AS6EL6TAR3JYIN5FNFFTNVWBNPFQ36Q6UULCZTOONU43QXUQKJMA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.637901635 (1252h26m19.776885829s) [noise=true , address=167.17.72.68:4669 , piece_id=TUI2SYKYEI4SVMG46LKN53ZRKCLB4AXKJNEMTVAWOU35TJZD2AKQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.934800787 (1252h26m20.073784972s) [noise=true , address=64.49.13.27:4967 , piece_id=TQRB662OLQAVXVXKB4X6U7PSPYQMUIMNEQLPKR2CXQGFZB7FSCWQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.939956558 (1252h26m20.078940752s) [noise=true , address=193.149.164.71:4298 , piece_id=NQ4TYVSF6AAV66PD7J55PP5LEIQEDCRHCY6PSMMI4RE52LNFJSPA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.940039554 (1252h26m20.079023737s) [noise=true , address=193.149.164.67:4997 , piece_id=7TRSFVVGUFVQIVXUEKC3RZMNMDZUHZFNNFAZIWVN25KN3TNVVGUQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941150174 (1252h26m20.080134359s) [noise=true , address=193.149.164.69:4071 , piece_id=IHOMNYUNJ57XQMN65I74JNYM7ILDRQ63N6BB7T7QAC2CGONXGPVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941411514 (1252h26m20.080395698s) [noise=true , address=193.149.164.66:4979 , piece_id=5WDHSGASPLS3BGJSWIZ5LCL24P67VTJTTWD273PIUC4QI4DHIYEA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.941849996 (1252h26m20.08083418s) [noise=true , address=193.149.164.70:4137 , piece_id=CO4C5KNHAOMMSX64NLCUJ5EV2GCFCEECB7DYRPBJLMW5ATNIEIVA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:00.960649371 (1252h26m20.099633565s) [noise=true , address=64.49.13.140:4507 , piece_id=XG5N7OU5GLI4P7SH5UKSJUVRVCUQB676275RUMKATBZJKJAZRKLQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:35:01.307829827 (1252h26m20.446814012s) [noise=true , address=77.90.31.17:4800 , piece_id=TW5YEGWX7COIVE3OR3G3U3XITBTEY5LAQQBQOZDXVWUSS2H4RKYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: <nil>
Apr 11 00:36:00.327591793 (1252h27m19.466575967s) [noise=true , address=145.79.227.19:4089 , piece_id=MOP6VLWI4PF3DHPRR34SWYETNKX6WYH4TD2EABDK5DF6MV6J5MWA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12bFi2opJ6FejAMwSL2qbqWoCk24QzSjT6xQYE8BytLgVJikZMT, address:145.79.227.19:4089): protocol: rpc: dial tcp 145.79.227.19:4089: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327637769 (1252h27m19.466621964s) [noise=true , address=145.79.227.52:4461 , piece_id=ZDDNKD3IHWX7LEXJHD6I4532MR4ICBMIE7S3IHHBIA7PRPD7DI3A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12fNrbSfErEQRnruJCJAVvkoNwQSPWnvNtxunHn6zrQegnNAkxA, address:145.79.227.52:4461): protocol: rpc: dial tcp 145.79.227.52:4461: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327731975 (1252h27m19.46671616s) [noise=true , address=145.79.227.13:4829 , piece_id=PVDRSVZQFPMNGVGNOKL3KD65CNP3F7QAK4CTTVEFFKOIEFKFJQ6A, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12aExoQdioFcK1qB46HjSQA6jany7ZxxgRYbBcBu75duXBEKWbi, address:145.79.227.13:4829): protocol: rpc: dial tcp 145.79.227.13:4829: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327830250 (1252h27m19.466814424s) [noise=true , address=145.79.227.32:4847 , piece_id=2BKKVHJNMBFEQWVCEADWLC4DZJB3G4ZYEW4R4THFY6J2PN6AGRSQ, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1JEqRG4b2YCdARm1eBnCgH4sAW4X7kRykNRZyTstpi8xavLqEg, address:145.79.227.32:4847): protocol: rpc: dial tcp 145.79.227.32:4847: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.327875465 (1252h27m19.466859659s) [noise=true , address=145.79.227.40:4061 , piece_id=665BK63Z5EAMEX4KQL4QIEG2Y6UW4J7C2FIGAI2Z4C52NULVPRYA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:12KDgmvhgoRSTnByRrzSuw2N4iobi59Atdm2N2SsWZ4yoyfFHof, address:145.79.227.40:4061): protocol: rpc: dial tcp 145.79.227.40:4061: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328146733 (1252h27m19.467130927s) [noise=true , address=145.79.227.36:4542 , piece_id=B3ONGCMUTNCX3Z3S222GIZKV2ZI26L2DFVNVMX3CYCO6DGCN65GA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:129FwaCA1FbzJbmRNzhQeVgygicJYHWkSS9fLnJbAG7KZPpokLU, address:145.79.227.36:4542): protocol: rpc: dial tcp 145.79.227.36:4542: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328250037 (1252h27m19.467234211s) [noise=true , address=145.79.227.29:4973 , piece_id=2YQTFAU66N6WFOQMANYBTDJ5HPWJOOR3TUR6ABFP2JXC2AHA2IHA, seg_pos={0 0}, upload=1101192]: Done uploading piece. err: ecclient: upload failed (node:1dCvsMCm5ZRmWVqP1zr9noHVuPeyrXHjpRvdHnKoAZqqt6knx, address:145.79.227.29:4973): protocol: rpc: dial tcp 145.79.227.29:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Apr 11 00:36:00.328267800 (1252h27m19.467251985s) [seg_pos={0 0}, upload=1101192]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil>
Apr 11 00:36:00.349355634 (1252h27m19.488339828s) [upload=1101192]: Done uploading segments...
Apr 11 00:36:00.349374158 (1252h27m19.488358352s) [upload=1101192]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11)
Duration: 1.669328481s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
116.573449ms remote:
"tcp:136.0.77.3:443" response headers:
X-Amz-Request-Id=18A52692A70DC5BD Date=Sat, 11 Apr 2026 00:40:39 GMT Accept-Ranges=bytes Content-Length=274 Server=Storj X-Request-Id=2ug3omrRTS1 X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/53e7603cd23d347f
FAIL (2026-04-11 00:40:39 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A52692A70DC5BD</RequestId><HostId></HostId></Error>"
Duration: 116.585952ms
edge
gateway
vhost
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.660083928s trace:
https://jaeger.dev.storj.io/trace/23afef2367e31b52
FAIL (2026-04-11 00:40:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.660106742s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
12.016986ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin Date=Sat, 11 Apr 2026 00:40:36 GMT Content-Type=application/xml X-Amz-Request-Id=18A5269208258264 X-Request-Id=8b3o4XYzRvp X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=284 target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/c2aeb67d7e0d922
FAIL (2026-04-11 00:40:36 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A5269208258264</RequestId><HostId></HostId></Error>"
Duration: 12.033297ms
edge
gateway
vhost
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
132.922292ms remote:
"tcp:136.0.77.3:443" response headers:
X-Amz-Request-Id=18A5269109B008F9 Content-Security-Policy=block-all-mixed-content Server=Storj X-Request-Id=8vcwsWaCaum X-Xss-Protection=1; mode=block Date=Sat, 11 Apr 2026 00:40:32 GMT Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/65f909a597fd002f
FAIL (2026-04-11 00:40:32 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A5269109B008F9</RequestId><HostId></HostId></Error>"
Duration: 132.938412ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
1.23970104s trace:
https://jaeger.dev.storj.io/trace/75735e15af9b9b94
FAIL (2026-04-11 00:40:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.239750423s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
106.573972ms remote:
"tcp:136.0.77.3:443" response headers:
Vary=Origin X-Request-Id=3TfarQJ7tgA Date=Sat, 11 Apr 2026 00:39:59 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml X-Amz-Request-Id=18A52689684F1F82 X-Xss-Protection=1; mode=block Server=Storj target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/285522c930c32fa7
FAIL (2026-04-11 00:39:59 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A52689684F1F82</RequestId><HostId></HostId></Error>"
Duration: 106.594209ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
112.451706ms remote:
"tcp:136.0.77.3:443" response headers:
X-Xss-Protection=1; mode=block Date=Sat, 11 Apr 2026 00:39:57 GMT Content-Length=274 Content-Type=application/xml Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj Vary=Origin X-Amz-Request-Id=18A52688D37C03B7 X-Request-Id=8LFJGney7dz target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/67b383e1345c5b49
FAIL (2026-04-11 00:39:57 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A52688D37C03B7</RequestId><HostId></HostId></Error>"
Duration: 112.470131ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.750461143s trace:
https://jaeger.dev.storj.io/trace/23afef2367e31b52
FAIL (2026-04-11 00:39:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.750481852s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
8.028303ms remote:
"tcp:45.38.24.2:443" response headers:
X-Xss-Protection=1; mode=block Vary=Origin X-Amz-Request-Id=18A52687D4430CC6 Date=Sat, 11 Apr 2026 00:39:52 GMT Accept-Ranges=bytes Content-Length=284 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Request-Id=56MdGJ8DHAP target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/7fa28e05c61d76d
FAIL (2026-04-11 00:39:52 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>18A52687D4430CC6</RequestId><HostId></HostId></Error>"
Duration: 8.046106ms
edge
gateway
vhost
us-select-1
|