Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......] duration:
497.488082ms trace:
https://jaeger.dev.storj.io/trace/74a421320e9af30f
FAIL (2026-02-24 18:17:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 497.511747ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
1m0.000961732s trace:
https://jaeger.dev.storj.io/trace/4e6a4f46effbd1a9
FAIL (2026-02-24 18:17:54 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Feb 24 18:16:54.111781377 (166h8m13.250765591s) [upload=146149]: Starting upload Feb 24 18:16:54.111867287 (166h8m13.250851502s) [upload=146149]: Done starting upload Feb 24 18:16:54.111963216 (166h8m13.250947431s) [upload=146149]: Uploading segments... Feb 24 18:16:54.112058745 (166h8m13.251042949s) [upload=146149]: Got next segment. Inline: true Feb 24 18:16:54.112065818 (166h8m13.251050032s) [upload=146149]: Waiting for error group managing segments... Feb 24 18:16:54.112152771 (166h8m13.251136985s) [upload=146149]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:16:54.112164263 (166h8m13.251148476s) [upload=146149]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Feb 24 18:16:54.112173338 (166h8m13.251157593s) [upload=146149]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Feb 24 18:17:54.099714608 (166h9m13.238698792s) [upload=146149]: Done uploading segments... Feb 24 18:17:54.099741959 (166h9m13.238726143s) [upload=146149]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.000979335s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
1m0.001532761s trace:
https://jaeger.dev.storj.io/trace/b4234bf01f9abd0
FAIL (2026-02-24 18:17:50 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001547979s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..XX........................................................] duration:
2.259265906s trace:
https://jaeger.dev.storj.io/trace/24b4edf9256675dc
FAIL (2026-02-24 18:17:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
7EICWCWMEABP3ENF5Q, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.249727323 (165h26m16.388711507s) [noise=true , address=81.171.14.70:4036 , piece_id=N4AIPU6IKD3HPSXHROJSWHFLAN573JOD6W6TUTV6FZQEMWVYCSHA, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250011063 (165h26m16.388995257s) [noise=true , address=185.167.204.206:4070 , piece_id=OQP5Z7ZJ3K76NJK3QSMAZEJTI6HFT7FUHT5E5IXAX4RJLHMB4I7A, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250290094 (165h26m16.389274278s) [noise=true , address=185.167.204.206:4056 , piece_id=YJJKP7ZHN6XESSLSHRTNWBCVU7ZLDNPQUUPE7V2ZMOLR4UJ2WX5Q, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250587089 (165h26m16.389571283s) [noise=true , address=185.167.204.205:4026 , piece_id=JMBMNNNQGEPQ5I5PGG6ZLNP2SAJEOPXM3KVTB6NMVYBVTBU27I2Q, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250850792 (165h26m16.389834976s) [noise=true , address=81.171.14.70:4030 , piece_id=B4LMP6JJEI7KFLUTDP4LTLX3DKB6EJASZLQLHXX4PHXOYDW6V2LA, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.724619104 (165h26m16.863603319s) [upload=145547]: Next returned nil segment
Feb 24 17:34:57.724631829 (165h26m16.863616042s) [upload=145547]: Waiting for error group managing segments...
Feb 24 17:34:57.901571686 (165h26m17.04055587s) [noise=true , address=89.149.201.147:4011 , piece_id=UYS6AJGSLDCVZ2R2YYZQP7W7S3RNKGW5RPPPCATQGULFW6HS7HDQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.908951002 (165h26m17.047935196s) [noise=true , address=89.149.201.147:4012 , piece_id=TSXMBWUIWR4S3CGSTJF6Y35EM22QFYSOB3HLGBV7LBZW4MKV7QFQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.915198182 (165h26m17.054182396s) [noise=true , address=185.167.204.205:4026 , piece_id=JMBMNNNQGEPQ5I5PGG6ZLNP2SAJEOPXM3KVTB6NMVYBVTBU27I2Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.915867082 (165h26m17.054851295s) [noise=true , address=185.167.204.205:4027 , piece_id=7AKTJIIAHEOOP3BNPWH4TB7ISZO7G6UM3JPYQLOF3MATWPZOEPCA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.916644374 (165h26m17.055628588s) [noise=true , address=185.167.204.205:4036 , piece_id=5UFBMTYMSM5VCPXZISA7PLK3B6WPTEHVTRIPMAUTEIFQ5CDRBQVQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.917614386 (165h26m17.0565986s) [noise=true , address=89.149.201.147:4023 , piece_id=NXX5UHIY23R5NVUM3QEMMFHYUVYFRRMLBZNUVELN3ZTAJSTCGUHA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.933365171 (165h26m17.072349365s) [noise=true , address=89.149.201.147:4014 , piece_id=IUSO2AFUBTVD4RUQXBY7GMLF6QX6U5UPS6J34Y2ZHSXMK4BK3WUA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.938423079 (165h26m17.077407294s) [noise=true , address=185.167.204.205:4004 , piece_id=O3OFG2BPNLEWVLL2FSPTDOKKVT7FUBEIISOWVRI7VD2ZDR5PBVNQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.964706541 (165h26m17.103690755s) [noise=true , address=185.167.204.206:4069 , piece_id=XAOTFHYPCHWOXE67HR5EB56FICFCTBGEOVE7KVSQV2FQ2NWDNKWA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.965170328 (165h26m17.104154512s) [noise=true , address=185.167.204.205:4012 , piece_id=Q5JTWCNRG4SBMAE4YCJKJHGDQPE7AOINO2LRLBRGXW5O36OB5WVA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.971952097 (165h26m17.11093631s) [noise=true , address=185.167.204.206:4053 , piece_id=O6BLFT5UE3IEWSHAAUTGYTBINPYSWFBTLWC4TYPW3XN5VPLBDKRQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.972322990 (165h26m17.111307173s) [noise=true , address=185.167.204.206:4066 , piece_id=5FVO2YQGZV2WOAZ7T5H3LNJUNXXMK55OCZENG35DFEEWBLA5N5GQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.975251300 (165h26m17.114235483s) [noise=true , address=185.167.204.206:4041 , piece_id=DDMUQAEF7JPW6FR6LYCWH3AWY5HF4SA7IQCBOOKHOFAVZLWE23DQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.980987377 (165h26m17.11997159s) [noise=true , address=185.167.204.206:4070 , piece_id=OQP5Z7ZJ3K76NJK3QSMAZEJTI6HFT7FUHT5E5IXAX4RJLHMB4I7A, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.983946945 (165h26m17.122931129s) [noise=true , address=185.167.204.206:4056 , piece_id=YJJKP7ZHN6XESSLSHRTNWBCVU7ZLDNPQUUPE7V2ZMOLR4UJ2WX5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.989584816 (165h26m17.128569041s) [noise=true , address=89.149.201.147:4018 , piece_id=OEQOZB5IRSIY5CR5UJ5XXMZPDGH4XN6KQ7ZGCZMBYNBX6OSMOKQQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.995870059 (165h26m17.134854243s) [noise=true , address=89.149.201.147:4013 , piece_id=R6HK4UBOGRHIEOHSJJA7DRSVDRNF27QW4Z7EICWCWMEABP3ENF5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:49.889621672 (165h27m9.028605886s) [noise=true , address=81.171.14.70:4036 , piece_id=N4AIPU6IKD3HPSXHROJSWHFLAN573JOD6W6TUTV6FZQEMWVYCSHA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:50.581661013 (165h27m9.720645197s) [noise=true , address=81.171.14.70:4034 , piece_id=2B3TO6XY2NVRX5SQ5SUW6OEMLWZHYNPWHOHBL5EAWM2PEDV4KHHQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:53.182181244 (165h27m12.321165459s) [noise=true , address=81.171.14.70:4033 , piece_id=H6ZB652A5CNAUCNLXHRMXJKXRNBK2HGWENUY3AKCRSWMK2IHE3AA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:55.084212904 (165h27m14.223197098s) [noise=true , address=81.171.14.70:4041 , piece_id=XGTKMITYQPT67CNFDJJGZW3TWFNIFGMGZYBBV7RDGNSPJGBBKX5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:56.826408639 (165h27m15.965392853s) [noise=true , address=81.171.14.70:4047 , piece_id=SLTCRDUZZ74TF7UUKKA3ZW4DV4RO7VO7CW37RZ3KOTXCPNEBWVAQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 5.161.89.252:51760->81.171.14.70:4047: use of closed network connection; write tcp 5.161.89.252:51760->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:51760->81.171.14.70:4047: use of closed network connection
Feb 24 17:35:56.826540285 (165h27m15.965524498s) [noise=true , address=81.171.14.70:4037 , piece_id=Z7IPSTK2E3WQ6RPTB43HFST52BECH7VRHLKPV6V5FWOMJ7YNWFCA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: read tcp 5.161.89.252:61326->81.171.14.70:4037: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 24 17:35:56.826589036 (165h27m15.96557326s) [noise=true , address=81.171.14.70:4030 , piece_id=B4LMP6JJEI7KFLUTDP4LTLX3DKB6EJASZLQLHXX4PHXOYDW6V2LA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 5.161.89.252:54788->81.171.14.70:4030: use of closed network connection; write tcp 5.161.89.252:54788->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:54788->81.171.14.70:4030: use of closed network connection
Feb 24 17:35:56.826781545 (165h27m15.965765759s) [noise=true , address=81.171.14.70:4042 , piece_id=FP4T4TB3C5K2HORTY55FWKP4AQYJYTT3AIQE5ERTDG6IVEKLXB6Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 5.161.89.252:16670->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 24 17:35:56.826879228 (165h27m15.965863452s) [noise=true , address=81.171.14.70:4048 , piece_id=S5QG4SXCIEBZAJCPNY3GYLWO3OA3LXPMGDZWLIVTJMC5WZ7MIT5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF
Feb 24 17:35:56.827059424 (165h27m15.966043638s) [noise=true , address=81.171.14.70:4046 , piece_id=ROYQDMKWPNWJWK2ZXNO46P75STGAHHN7OHTR2ABQNBGCCYKTABQQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 5.161.89.252:12860->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 24 17:35:56.827234200 (165h27m15.966218415s) [noise=true , address=81.171.14.70:4029 , piece_id=ANBM4PEUW4YPPXUYUB2VAKXMDBEXBD7CO42CXEHAJ2W5NDNXL4WQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 5.161.89.252:10314->81.171.14.70:4029: use of closed network connection; write tcp 5.161.89.252:10314->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10314->81.171.14.70:4029: use of closed network connection
Feb 24 17:35:56.827285327 (165h27m15.966269551s) [seg_pos={0 0}, upload=145547]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Feb 24 17:35:56.937070410 (165h27m16.076054634s) [upload=145547]: Done uploading segments...
Feb 24 17:35:56.937143686 (165h27m16.076127911s) [upload=145547]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 2.259285573s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
30.000456709s trace:
https://jaeger.dev.storj.io/trace/bf9e139aad9c61b
FAIL (2026-02-24 18:17:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000466318s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
1m0.000884138s trace:
https://jaeger.dev.storj.io/trace/4f1ffaf25f5601f
FAIL (2026-02-24 18:17:44 UTC)
operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 24 18:16:44.149863613 (166h8m3.288847807s) [upload=146148]: Starting upload Feb 24 18:16:44.149908968 (166h8m3.288893151s) [upload=146148]: Done starting upload Feb 24 18:16:44.150012350 (166h8m3.288996534s) [upload=146148]: Uploading segments... Feb 24 18:16:44.150155548 (166h8m3.289139742s) [upload=146148]: Got next segment. Inline: false Feb 24 18:16:44.150184873 (166h8m3.289169067s) [upload=146148]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:16:44.150188880 (166h8m3.289173064s) [upload=146148]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 24 18:17:44.133000984 (166h9m3.271985168s) [upload=146148]: Next segment err: context deadline exceeded Feb 24 18:17:44.133357270 (166h9m3.272341464s) [upload=146148]: Done uploading segments... Feb 24 18:17:44.133428423 (166h9m3.272412617s) [upload=146148]: Upload finished. err: context deadline exceeded
Duration: 1m0.000892504s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
22.869667ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Date=Tue, 24 Feb 2026 18:17:40 GMT Accept-Ranges=bytes Server=Storj Vary=Origin X-Amz-Request-Id=1897418EE40F9FE9 X-Request-Id=JTFoEyhPCVt X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/2d997979ab538224
FAIL (2026-02-24 18:17:40 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897418EE40F9FE9</RequestId><HostId></HostId></Error>"
Duration: 22.881297ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
30.001077561s trace:
https://jaeger.dev.storj.io/trace/577f5f3af5f9be26
FAIL (2026-02-24 18:17:37 UTC)
operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.00109289s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
30.001255675s trace:
https://jaeger.dev.storj.io/trace/6691f010f341caed
FAIL (2026-02-24 18:17:36 UTC)
operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.001269811s
core
uplink
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
23.062297ms remote:
"tcp:136.0.77.3:443" response headers:
X-Request-Id=JU8GBSLajmq Date=Tue, 24 Feb 2026 18:17:35 GMT Accept-Ranges=bytes Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=1897418DDBDB810F X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/5286bce4bf1ef748
FAIL (2026-02-24 18:17:35 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897418DDBDB810F</RequestId><HostId></HostId></Error>"
Duration: 23.07966ms
edge
gateway
vhost
bgp
eu1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
28.170491ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=1897418C7D58F552 Date=Tue, 24 Feb 2026 18:17:30 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin X-Request-Id=LHWBq1B23yw X-Xss-Protection=1; mode=block target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/10a1e02cc9ea7a2c
FAIL (2026-02-24 18:17:30 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897418C7D58F552</RequestId><HostId></HostId></Error>"
Duration: 28.192452ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration:
30.001512052s trace:
https://jaeger.dev.storj.io/trace/4164ecda73cd4b64
FAIL (2026-02-24 18:17:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001526599s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
30.001177136s trace:
https://jaeger.dev.storj.io/trace/36e6060de93a169a
FAIL (2026-02-24 18:17:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001185761s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....] duration:
504.963297ms trace:
https://jaeger.dev.storj.io/trace/74a421320e9af30f
FAIL (2026-02-24 18:17:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 504.980078ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
30.002119366s trace:
https://jaeger.dev.storj.io/trace/52fdf95062b0a9a5
FAIL (2026-02-24 18:17:14 UTC)
operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 24 18:16:44.034282185 (166h8m3.173266399s) [upload=146147]: Starting upload Feb 24 18:16:44.034422367 (166h8m3.173406591s) [upload=146147]: Done starting upload Feb 24 18:16:44.034481979 (166h8m3.173466203s) [upload=146147]: Uploading segments... Feb 24 18:16:44.034806455 (166h8m3.173790669s) [upload=146147]: Got next segment. Inline: false Feb 24 18:16:44.034878890 (166h8m3.173863104s) [upload=146147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:16:44.034886454 (166h8m3.173870668s) [upload=146147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 24 18:17:14.021929894 (166h8m33.160914118s) [upload=146147]: Next segment err: context deadline exceeded Feb 24 18:17:14.022458502 (166h8m33.161442726s) [upload=146147]: Done uploading segments... Feb 24 18:17:14.022542238 (166h8m33.161526453s) [upload=146147]: Upload finished. err: context deadline exceeded
Duration: 30.002135427s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
13.533183ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Length=284 Content-Type=application/xml Vary=Origin X-Request-Id=ZiRc76v6o6G Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=1897418834FFA41A X-Xss-Protection=1; mode=block Date=Tue, 24 Feb 2026 18:17:11 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/5350ad7e476ad50d
FAIL (2026-02-24 18:17:11 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897418834FFA41A</RequestId><HostId></HostId></Error>"
Duration: 13.556678ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
30.000885357s trace:
https://jaeger.dev.storj.io/trace/631618f5ba0d6d5f
FAIL (2026-02-24 18:17:05 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Feb 24 18:16:35.920850259 (166h7m55.059834453s) [upload=146145]: Starting upload Feb 24 18:16:35.920894080 (166h7m55.059878264s) [upload=146145]: Done starting upload Feb 24 18:16:35.920941438 (166h7m55.059925623s) [upload=146145]: Uploading segments... Feb 24 18:16:35.920971365 (166h7m55.059955559s) [upload=146145]: Got next segment. Inline: true Feb 24 18:16:35.920975302 (166h7m55.059959496s) [upload=146145]: Waiting for error group managing segments... Feb 24 18:16:35.921015728 (166h7m55.059999912s) [upload=146145]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:16:35.921019455 (166h7m55.060003639s) [upload=146145]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Feb 24 18:16:35.921021940 (166h7m55.060006134s) [upload=146145]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Feb 24 18:17:05.907999053 (166h8m25.046983247s) [upload=146145]: Done uploading segments... Feb 24 18:17:05.908028268 (166h8m25.047012462s) [upload=146145]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.000900224s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
30.000715061s trace:
https://jaeger.dev.storj.io/trace/7878dedb9359c5d6
FAIL (2026-02-24 18:17:03 UTC)
operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.000749866s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
30.001011244s trace:
https://jaeger.dev.storj.io/trace/504f4119f068123
FAIL (2026-02-24 18:17:00 UTC)
operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.001033335s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...XX.......................................................] duration:
2.183820871s trace:
https://jaeger.dev.storj.io/trace/24b4edf9256675dc
FAIL (2026-02-24 18:16:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
7EICWCWMEABP3ENF5Q, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.249727323 (165h26m16.388711507s) [noise=true , address=81.171.14.70:4036 , piece_id=N4AIPU6IKD3HPSXHROJSWHFLAN573JOD6W6TUTV6FZQEMWVYCSHA, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250011063 (165h26m16.388995257s) [noise=true , address=185.167.204.206:4070 , piece_id=OQP5Z7ZJ3K76NJK3QSMAZEJTI6HFT7FUHT5E5IXAX4RJLHMB4I7A, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250290094 (165h26m16.389274278s) [noise=true , address=185.167.204.206:4056 , piece_id=YJJKP7ZHN6XESSLSHRTNWBCVU7ZLDNPQUUPE7V2ZMOLR4UJ2WX5Q, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250587089 (165h26m16.389571283s) [noise=true , address=185.167.204.205:4026 , piece_id=JMBMNNNQGEPQ5I5PGG6ZLNP2SAJEOPXM3KVTB6NMVYBVTBU27I2Q, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250850792 (165h26m16.389834976s) [noise=true , address=81.171.14.70:4030 , piece_id=B4LMP6JJEI7KFLUTDP4LTLX3DKB6EJASZLQLHXX4PHXOYDW6V2LA, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.724619104 (165h26m16.863603319s) [upload=145547]: Next returned nil segment
Feb 24 17:34:57.724631829 (165h26m16.863616042s) [upload=145547]: Waiting for error group managing segments...
Feb 24 17:34:57.901571686 (165h26m17.04055587s) [noise=true , address=89.149.201.147:4011 , piece_id=UYS6AJGSLDCVZ2R2YYZQP7W7S3RNKGW5RPPPCATQGULFW6HS7HDQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.908951002 (165h26m17.047935196s) [noise=true , address=89.149.201.147:4012 , piece_id=TSXMBWUIWR4S3CGSTJF6Y35EM22QFYSOB3HLGBV7LBZW4MKV7QFQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.915198182 (165h26m17.054182396s) [noise=true , address=185.167.204.205:4026 , piece_id=JMBMNNNQGEPQ5I5PGG6ZLNP2SAJEOPXM3KVTB6NMVYBVTBU27I2Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.915867082 (165h26m17.054851295s) [noise=true , address=185.167.204.205:4027 , piece_id=7AKTJIIAHEOOP3BNPWH4TB7ISZO7G6UM3JPYQLOF3MATWPZOEPCA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.916644374 (165h26m17.055628588s) [noise=true , address=185.167.204.205:4036 , piece_id=5UFBMTYMSM5VCPXZISA7PLK3B6WPTEHVTRIPMAUTEIFQ5CDRBQVQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.917614386 (165h26m17.0565986s) [noise=true , address=89.149.201.147:4023 , piece_id=NXX5UHIY23R5NVUM3QEMMFHYUVYFRRMLBZNUVELN3ZTAJSTCGUHA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.933365171 (165h26m17.072349365s) [noise=true , address=89.149.201.147:4014 , piece_id=IUSO2AFUBTVD4RUQXBY7GMLF6QX6U5UPS6J34Y2ZHSXMK4BK3WUA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.938423079 (165h26m17.077407294s) [noise=true , address=185.167.204.205:4004 , piece_id=O3OFG2BPNLEWVLL2FSPTDOKKVT7FUBEIISOWVRI7VD2ZDR5PBVNQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.964706541 (165h26m17.103690755s) [noise=true , address=185.167.204.206:4069 , piece_id=XAOTFHYPCHWOXE67HR5EB56FICFCTBGEOVE7KVSQV2FQ2NWDNKWA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.965170328 (165h26m17.104154512s) [noise=true , address=185.167.204.205:4012 , piece_id=Q5JTWCNRG4SBMAE4YCJKJHGDQPE7AOINO2LRLBRGXW5O36OB5WVA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.971952097 (165h26m17.11093631s) [noise=true , address=185.167.204.206:4053 , piece_id=O6BLFT5UE3IEWSHAAUTGYTBINPYSWFBTLWC4TYPW3XN5VPLBDKRQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.972322990 (165h26m17.111307173s) [noise=true , address=185.167.204.206:4066 , piece_id=5FVO2YQGZV2WOAZ7T5H3LNJUNXXMK55OCZENG35DFEEWBLA5N5GQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.975251300 (165h26m17.114235483s) [noise=true , address=185.167.204.206:4041 , piece_id=DDMUQAEF7JPW6FR6LYCWH3AWY5HF4SA7IQCBOOKHOFAVZLWE23DQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.980987377 (165h26m17.11997159s) [noise=true , address=185.167.204.206:4070 , piece_id=OQP5Z7ZJ3K76NJK3QSMAZEJTI6HFT7FUHT5E5IXAX4RJLHMB4I7A, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.983946945 (165h26m17.122931129s) [noise=true , address=185.167.204.206:4056 , piece_id=YJJKP7ZHN6XESSLSHRTNWBCVU7ZLDNPQUUPE7V2ZMOLR4UJ2WX5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.989584816 (165h26m17.128569041s) [noise=true , address=89.149.201.147:4018 , piece_id=OEQOZB5IRSIY5CR5UJ5XXMZPDGH4XN6KQ7ZGCZMBYNBX6OSMOKQQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.995870059 (165h26m17.134854243s) [noise=true , address=89.149.201.147:4013 , piece_id=R6HK4UBOGRHIEOHSJJA7DRSVDRNF27QW4Z7EICWCWMEABP3ENF5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:49.889621672 (165h27m9.028605886s) [noise=true , address=81.171.14.70:4036 , piece_id=N4AIPU6IKD3HPSXHROJSWHFLAN573JOD6W6TUTV6FZQEMWVYCSHA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:50.581661013 (165h27m9.720645197s) [noise=true , address=81.171.14.70:4034 , piece_id=2B3TO6XY2NVRX5SQ5SUW6OEMLWZHYNPWHOHBL5EAWM2PEDV4KHHQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:53.182181244 (165h27m12.321165459s) [noise=true , address=81.171.14.70:4033 , piece_id=H6ZB652A5CNAUCNLXHRMXJKXRNBK2HGWENUY3AKCRSWMK2IHE3AA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:55.084212904 (165h27m14.223197098s) [noise=true , address=81.171.14.70:4041 , piece_id=XGTKMITYQPT67CNFDJJGZW3TWFNIFGMGZYBBV7RDGNSPJGBBKX5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:56.826408639 (165h27m15.965392853s) [noise=true , address=81.171.14.70:4047 , piece_id=SLTCRDUZZ74TF7UUKKA3ZW4DV4RO7VO7CW37RZ3KOTXCPNEBWVAQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 5.161.89.252:51760->81.171.14.70:4047: use of closed network connection; write tcp 5.161.89.252:51760->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:51760->81.171.14.70:4047: use of closed network connection
Feb 24 17:35:56.826540285 (165h27m15.965524498s) [noise=true , address=81.171.14.70:4037 , piece_id=Z7IPSTK2E3WQ6RPTB43HFST52BECH7VRHLKPV6V5FWOMJ7YNWFCA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: read tcp 5.161.89.252:61326->81.171.14.70:4037: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 24 17:35:56.826589036 (165h27m15.96557326s) [noise=true , address=81.171.14.70:4030 , piece_id=B4LMP6JJEI7KFLUTDP4LTLX3DKB6EJASZLQLHXX4PHXOYDW6V2LA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 5.161.89.252:54788->81.171.14.70:4030: use of closed network connection; write tcp 5.161.89.252:54788->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:54788->81.171.14.70:4030: use of closed network connection
Feb 24 17:35:56.826781545 (165h27m15.965765759s) [noise=true , address=81.171.14.70:4042 , piece_id=FP4T4TB3C5K2HORTY55FWKP4AQYJYTT3AIQE5ERTDG6IVEKLXB6Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 5.161.89.252:16670->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 24 17:35:56.826879228 (165h27m15.965863452s) [noise=true , address=81.171.14.70:4048 , piece_id=S5QG4SXCIEBZAJCPNY3GYLWO3OA3LXPMGDZWLIVTJMC5WZ7MIT5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF
Feb 24 17:35:56.827059424 (165h27m15.966043638s) [noise=true , address=81.171.14.70:4046 , piece_id=ROYQDMKWPNWJWK2ZXNO46P75STGAHHN7OHTR2ABQNBGCCYKTABQQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 5.161.89.252:12860->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 24 17:35:56.827234200 (165h27m15.966218415s) [noise=true , address=81.171.14.70:4029 , piece_id=ANBM4PEUW4YPPXUYUB2VAKXMDBEXBD7CO42CXEHAJ2W5NDNXL4WQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 5.161.89.252:10314->81.171.14.70:4029: use of closed network connection; write tcp 5.161.89.252:10314->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10314->81.171.14.70:4029: use of closed network connection
Feb 24 17:35:56.827285327 (165h27m15.966269551s) [seg_pos={0 0}, upload=145547]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Feb 24 17:35:56.937070410 (165h27m16.076054634s) [upload=145547]: Done uploading segments...
Feb 24 17:35:56.937143686 (165h27m16.076127911s) [upload=145547]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 2.183844685s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
23.083488ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin X-Request-Id=T2MD9ME92B9 Accept-Ranges=bytes Content-Type=application/xml X-Amz-Request-Id=18974184C65CAC99 X-Xss-Protection=1; mode=block Date=Tue, 24 Feb 2026 18:16:56 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/1e1f2f4883501057
FAIL (2026-02-24 18:16:56 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18974184C65CAC99</RequestId><HostId></HostId></Error>"
Duration: 23.099308ms
edge
gateway
vhost
bgp
ap1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....] duration:
516.119521ms trace:
https://jaeger.dev.storj.io/trace/74a421320e9af30f
FAIL (2026-02-24 18:16:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 516.144557ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
1m0.002019708s trace:
https://jaeger.dev.storj.io/trace/5622faa81d4aaa95
FAIL (2026-02-24 18:16:50 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.002035457s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
22.239832ms remote:
"tcp:136.0.77.3:443" response headers:
Content-Length=274 Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=18974182596B8975 X-Request-Id=YtMucHtc6qG X-Xss-Protection=1; mode=block Content-Type=application/xml Vary=Origin Date=Tue, 24 Feb 2026 18:16:46 GMT Accept-Ranges=bytes target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/1c7648e82cb71b2e
FAIL (2026-02-24 18:16:46 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>18974182596B8975</RequestId><HostId></HostId></Error>"
Duration: 22.269005ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
1m0.001337585s trace:
https://jaeger.dev.storj.io/trace/2350aec6255e3a40
FAIL (2026-02-24 18:16:45 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Feb 24 18:15:45.155399918 (166h7m4.294384123s) [upload=146134]: Starting upload Feb 24 18:15:45.155461244 (166h7m4.294445457s) [upload=146134]: Done starting upload Feb 24 18:15:45.155521926 (166h7m4.29450613s) [upload=146134]: Uploading segments... Feb 24 18:15:45.155560178 (166h7m4.294544392s) [upload=146134]: Got next segment. Inline: true Feb 24 18:15:45.155565608 (166h7m4.294549822s) [upload=146134]: Waiting for error group managing segments... Feb 24 18:15:45.155606895 (166h7m4.294591099s) [upload=146134]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:15:45.155612235 (166h7m4.29459645s) [upload=146134]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Feb 24 18:15:45.155615962 (166h7m4.294600177s) [upload=146134]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Feb 24 18:16:45.143053247 (166h8m4.282037461s) [upload=146134]: Done uploading segments... Feb 24 18:16:45.143093953 (166h8m4.282078178s) [upload=146134]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.001354046s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
1m0.000690908s trace:
https://jaeger.dev.storj.io/trace/39f6b27bf7783b74
FAIL (2026-02-24 18:16:44 UTC)
operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 24 18:15:44.095578597 (166h7m3.234562811s) [upload=146133]: Starting upload Feb 24 18:15:44.095651162 (166h7m3.234635436s) [upload=146133]: Done starting upload Feb 24 18:15:44.095693131 (166h7m3.234677345s) [upload=146133]: Uploading segments... Feb 24 18:15:44.096088068 (166h7m3.235072283s) [upload=146133]: Got next segment. Inline: false Feb 24 18:15:44.096147860 (166h7m3.235132085s) [upload=146133]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:15:44.096153952 (166h7m3.235138166s) [upload=146133]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 24 18:16:44.080360292 (166h8m3.219344506s) [upload=146133]: Next segment err: context deadline exceeded Feb 24 18:16:44.080656626 (166h8m3.219640839s) [upload=146133]: Done uploading segments... Feb 24 18:16:44.080713521 (166h8m3.219697735s) [upload=146133]: Upload finished. err: context deadline exceeded
Duration: 1m0.000700857s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
30.000606859s trace:
https://jaeger.dev.storj.io/trace/51cdafe2e2f421eb
FAIL (2026-02-24 18:16:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000624862s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
30.001035226s trace:
https://jaeger.dev.storj.io/trace/1fd9a9841faf6b71
FAIL (2026-02-24 18:16:43 UTC)
operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 24 18:16:13.990436336 (166h7m33.12942055s) [upload=146141]: Starting upload Feb 24 18:16:13.990523520 (166h7m33.129507744s) [upload=146141]: Done starting upload Feb 24 18:16:13.990711100 (166h7m33.129695313s) [upload=146141]: Uploading segments... Feb 24 18:16:13.990954926 (166h7m33.129939139s) [upload=146141]: Got next segment. Inline: false Feb 24 18:16:13.991040475 (166h7m33.130024689s) [upload=146141]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:16:13.991047218 (166h7m33.130031431s) [upload=146141]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 24 18:16:43.964639063 (166h8m3.103623257s) [upload=146141]: Next segment err: context deadline exceeded Feb 24 18:16:43.964909558 (166h8m3.103893752s) [upload=146141]: Done uploading segments... Feb 24 18:16:43.964963299 (166h8m3.103947493s) [upload=146141]: Upload finished. err: context deadline exceeded
Duration: 30.001054542s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
26.455876ms remote:
"tcp:136.0.77.3:443" response headers:
X-Amz-Request-Id=1897417F736409B6 X-Request-Id=GCiPg9wx4Qk X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Date=Tue, 24 Feb 2026 18:16:34 GMT Accept-Ranges=bytes Content-Length=274 Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/33a88e36e864085e
FAIL (2026-02-24 18:16:34 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897417F736409B6</RequestId><HostId></HostId></Error>"
Duration: 26.475281ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
30.001268265s trace:
https://jaeger.dev.storj.io/trace/3f0459ff74543bf1
FAIL (2026-02-24 18:16:32 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Feb 24 18:16:02.879013083 (166h7m22.017997268s) [upload=146138]: Starting upload Feb 24 18:16:02.879051285 (166h7m22.018035479s) [upload=146138]: Done starting upload Feb 24 18:16:02.879126126 (166h7m22.018110309s) [upload=146138]: Uploading segments... Feb 24 18:16:02.879159256 (166h7m22.01814345s) [upload=146138]: Got next segment. Inline: true Feb 24 18:16:02.879162964 (166h7m22.018147148s) [upload=146138]: Waiting for error group managing segments... Feb 24 18:16:02.879204531 (166h7m22.018188725s) [upload=146138]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:16:02.879209380 (166h7m22.018193565s) [upload=146138]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Feb 24 18:16:02.879224378 (166h7m22.018208572s) [upload=146138]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Feb 24 18:16:32.866582907 (166h7m52.005567101s) [upload=146138]: Done uploading segments... Feb 24 18:16:32.866612002 (166h7m52.005596196s) [upload=146138]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.001282802s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
30.001801572s trace:
https://jaeger.dev.storj.io/trace/1a15b60d2d3e0bbb
FAIL (2026-02-24 18:16:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001820628s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
22.39938ms remote:
"tcp:136.0.77.3:443" response headers:
Server=Storj Vary=Origin X-Amz-Request-Id=1897417C7BF6524C X-Request-Id=UgjiG7cbSr5 Content-Length=274 Content-Security-Policy=block-all-mixed-content X-Xss-Protection=1; mode=block Date=Tue, 24 Feb 2026 18:16:21 GMT Accept-Ranges=bytes Content-Type=application/xml target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/14f5ca6ad0f647c4
FAIL (2026-02-24 18:16:21 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897417C7BF6524C</RequestId><HostId></HostId></Error>"
Duration: 22.415319ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
30.001491942s trace:
https://jaeger.dev.storj.io/trace/7604a75a812a37d7
FAIL (2026-02-24 18:16:20 UTC)
operation: Commit uplink: metaclient: context deadline exceeded
Duration: 30.001510728s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...] duration:
532.893208ms trace:
https://jaeger.dev.storj.io/trace/74a421320e9af30f
FAIL (2026-02-24 18:16:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 532.912203ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
30.001059139s trace:
https://jaeger.dev.storj.io/trace/36f0213c17eacfe
FAIL (2026-02-24 18:16:19 UTC)
operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.001076221s
core
uplink
us1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)antiblip:
2 out of 60 is failed (threshold=6) [..........................................................XX] duration:
2m0.001332223s remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6c5e83f45e13d09f
FAIL (2026-02-24 18:16:19 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 2m0.001347973s
edge
linksharing
bgp
us1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
24.08629ms remote:
"tcp:136.0.77.3:443" response headers:
X-Request-Id=RZdWdUYFw7J X-Xss-Protection=1; mode=block Date=Tue, 24 Feb 2026 18:16:19 GMT Accept-Ranges=bytes Server=Storj X-Amz-Request-Id=1897417BF3FE6FC5 Content-Length=274 Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Vary=Origin target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/a76e39e466312f9
FAIL (2026-02-24 18:16:19 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897417BF3FE6FC5</RequestId><HostId></HostId></Error>"
Duration: 24.104774ms
edge
gateway
vhost
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
11.581709ms remote:
"tcp:45.38.24.2:443" response headers:
Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj Vary=Origin X-Amz-Request-Id=1897417BD423E3FE X-Request-Id=8ZRRmhdeDbq X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=284 Date=Tue, 24 Feb 2026 18:16:18 GMT target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 trace:
https://jaeger.dev.storj.io/trace/65883fabff4ce2c3
FAIL (2026-02-24 18:16:18 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897417BD423E3FE</RequestId><HostId></HostId></Error>"
Duration: 11.597898ms
edge
gateway
vhost
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....XX......................................................] duration:
2.179411345s trace:
https://jaeger.dev.storj.io/trace/24b4edf9256675dc
FAIL (2026-02-24 18:15:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
7EICWCWMEABP3ENF5Q, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.249727323 (165h26m16.388711507s) [noise=true , address=81.171.14.70:4036 , piece_id=N4AIPU6IKD3HPSXHROJSWHFLAN573JOD6W6TUTV6FZQEMWVYCSHA, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250011063 (165h26m16.388995257s) [noise=true , address=185.167.204.206:4070 , piece_id=OQP5Z7ZJ3K76NJK3QSMAZEJTI6HFT7FUHT5E5IXAX4RJLHMB4I7A, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250290094 (165h26m16.389274278s) [noise=true , address=185.167.204.206:4056 , piece_id=YJJKP7ZHN6XESSLSHRTNWBCVU7ZLDNPQUUPE7V2ZMOLR4UJ2WX5Q, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250587089 (165h26m16.389571283s) [noise=true , address=185.167.204.205:4026 , piece_id=JMBMNNNQGEPQ5I5PGG6ZLNP2SAJEOPXM3KVTB6NMVYBVTBU27I2Q, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.250850792 (165h26m16.389834976s) [noise=true , address=81.171.14.70:4030 , piece_id=B4LMP6JJEI7KFLUTDP4LTLX3DKB6EJASZLQLHXX4PHXOYDW6V2LA, seg_pos={0 0}, upload=145547]: Uploading piece...
Feb 24 17:34:57.724619104 (165h26m16.863603319s) [upload=145547]: Next returned nil segment
Feb 24 17:34:57.724631829 (165h26m16.863616042s) [upload=145547]: Waiting for error group managing segments...
Feb 24 17:34:57.901571686 (165h26m17.04055587s) [noise=true , address=89.149.201.147:4011 , piece_id=UYS6AJGSLDCVZ2R2YYZQP7W7S3RNKGW5RPPPCATQGULFW6HS7HDQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.908951002 (165h26m17.047935196s) [noise=true , address=89.149.201.147:4012 , piece_id=TSXMBWUIWR4S3CGSTJF6Y35EM22QFYSOB3HLGBV7LBZW4MKV7QFQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.915198182 (165h26m17.054182396s) [noise=true , address=185.167.204.205:4026 , piece_id=JMBMNNNQGEPQ5I5PGG6ZLNP2SAJEOPXM3KVTB6NMVYBVTBU27I2Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.915867082 (165h26m17.054851295s) [noise=true , address=185.167.204.205:4027 , piece_id=7AKTJIIAHEOOP3BNPWH4TB7ISZO7G6UM3JPYQLOF3MATWPZOEPCA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.916644374 (165h26m17.055628588s) [noise=true , address=185.167.204.205:4036 , piece_id=5UFBMTYMSM5VCPXZISA7PLK3B6WPTEHVTRIPMAUTEIFQ5CDRBQVQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.917614386 (165h26m17.0565986s) [noise=true , address=89.149.201.147:4023 , piece_id=NXX5UHIY23R5NVUM3QEMMFHYUVYFRRMLBZNUVELN3ZTAJSTCGUHA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.933365171 (165h26m17.072349365s) [noise=true , address=89.149.201.147:4014 , piece_id=IUSO2AFUBTVD4RUQXBY7GMLF6QX6U5UPS6J34Y2ZHSXMK4BK3WUA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.938423079 (165h26m17.077407294s) [noise=true , address=185.167.204.205:4004 , piece_id=O3OFG2BPNLEWVLL2FSPTDOKKVT7FUBEIISOWVRI7VD2ZDR5PBVNQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.964706541 (165h26m17.103690755s) [noise=true , address=185.167.204.206:4069 , piece_id=XAOTFHYPCHWOXE67HR5EB56FICFCTBGEOVE7KVSQV2FQ2NWDNKWA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.965170328 (165h26m17.104154512s) [noise=true , address=185.167.204.205:4012 , piece_id=Q5JTWCNRG4SBMAE4YCJKJHGDQPE7AOINO2LRLBRGXW5O36OB5WVA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.971952097 (165h26m17.11093631s) [noise=true , address=185.167.204.206:4053 , piece_id=O6BLFT5UE3IEWSHAAUTGYTBINPYSWFBTLWC4TYPW3XN5VPLBDKRQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.972322990 (165h26m17.111307173s) [noise=true , address=185.167.204.206:4066 , piece_id=5FVO2YQGZV2WOAZ7T5H3LNJUNXXMK55OCZENG35DFEEWBLA5N5GQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.975251300 (165h26m17.114235483s) [noise=true , address=185.167.204.206:4041 , piece_id=DDMUQAEF7JPW6FR6LYCWH3AWY5HF4SA7IQCBOOKHOFAVZLWE23DQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.980987377 (165h26m17.11997159s) [noise=true , address=185.167.204.206:4070 , piece_id=OQP5Z7ZJ3K76NJK3QSMAZEJTI6HFT7FUHT5E5IXAX4RJLHMB4I7A, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.983946945 (165h26m17.122931129s) [noise=true , address=185.167.204.206:4056 , piece_id=YJJKP7ZHN6XESSLSHRTNWBCVU7ZLDNPQUUPE7V2ZMOLR4UJ2WX5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.989584816 (165h26m17.128569041s) [noise=true , address=89.149.201.147:4018 , piece_id=OEQOZB5IRSIY5CR5UJ5XXMZPDGH4XN6KQ7ZGCZMBYNBX6OSMOKQQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:34:57.995870059 (165h26m17.134854243s) [noise=true , address=89.149.201.147:4013 , piece_id=R6HK4UBOGRHIEOHSJJA7DRSVDRNF27QW4Z7EICWCWMEABP3ENF5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:49.889621672 (165h27m9.028605886s) [noise=true , address=81.171.14.70:4036 , piece_id=N4AIPU6IKD3HPSXHROJSWHFLAN573JOD6W6TUTV6FZQEMWVYCSHA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:50.581661013 (165h27m9.720645197s) [noise=true , address=81.171.14.70:4034 , piece_id=2B3TO6XY2NVRX5SQ5SUW6OEMLWZHYNPWHOHBL5EAWM2PEDV4KHHQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:53.182181244 (165h27m12.321165459s) [noise=true , address=81.171.14.70:4033 , piece_id=H6ZB652A5CNAUCNLXHRMXJKXRNBK2HGWENUY3AKCRSWMK2IHE3AA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:55.084212904 (165h27m14.223197098s) [noise=true , address=81.171.14.70:4041 , piece_id=XGTKMITYQPT67CNFDJJGZW3TWFNIFGMGZYBBV7RDGNSPJGBBKX5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: <nil>
Feb 24 17:35:56.826408639 (165h27m15.965392853s) [noise=true , address=81.171.14.70:4047 , piece_id=SLTCRDUZZ74TF7UUKKA3ZW4DV4RO7VO7CW37RZ3KOTXCPNEBWVAQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:1bHiz99nonr7kwp4RLRvZdvKUJW3gVB3tx5vpDDbfijn2qrF2m, address:81.171.14.70:4047): protocol: write tcp 5.161.89.252:51760->81.171.14.70:4047: use of closed network connection; write tcp 5.161.89.252:51760->81.171.14.70:4047: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:51760->81.171.14.70:4047: use of closed network connection
Feb 24 17:35:56.826540285 (165h27m15.965524498s) [noise=true , address=81.171.14.70:4037 , piece_id=Z7IPSTK2E3WQ6RPTB43HFST52BECH7VRHLKPV6V5FWOMJ7YNWFCA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:1HSPWfW4L5gN2YoEV8rcqhQtVFv83M21nseT9MZSTgwJo76yGx, address:81.171.14.70:4037): protocol: read tcp 5.161.89.252:61326->81.171.14.70:4037: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 24 17:35:56.826589036 (165h27m15.96557326s) [noise=true , address=81.171.14.70:4030 , piece_id=B4LMP6JJEI7KFLUTDP4LTLX3DKB6EJASZLQLHXX4PHXOYDW6V2LA, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 5.161.89.252:54788->81.171.14.70:4030: use of closed network connection; write tcp 5.161.89.252:54788->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:54788->81.171.14.70:4030: use of closed network connection
Feb 24 17:35:56.826781545 (165h27m15.965765759s) [noise=true , address=81.171.14.70:4042 , piece_id=FP4T4TB3C5K2HORTY55FWKP4AQYJYTT3AIQE5ERTDG6IVEKLXB6Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: read tcp 5.161.89.252:16670->81.171.14.70:4042: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 24 17:35:56.826879228 (165h27m15.965863452s) [noise=true , address=81.171.14.70:4048 , piece_id=S5QG4SXCIEBZAJCPNY3GYLWO3OA3LXPMGDZWLIVTJMC5WZ7MIT5Q, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:12bbN67ZebXjF4iSyaWu4TDsG1XugTm1YdNHqRdRCbZ2WWRunRG, address:81.171.14.70:4048): protocol: expected piece hash; context deadline exceeded; EOF
Feb 24 17:35:56.827059424 (165h27m15.966043638s) [noise=true , address=81.171.14.70:4046 , piece_id=ROYQDMKWPNWJWK2ZXNO46P75STGAHHN7OHTR2ABQNBGCCYKTABQQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:1nYU5VwMDm67tjJYHi3jnC6PuKRapsXfziC8wdLroXe7CAJxQ7, address:81.171.14.70:4046): protocol: read tcp 5.161.89.252:12860->81.171.14.70:4046: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 24 17:35:56.827234200 (165h27m15.966218415s) [noise=true , address=81.171.14.70:4029 , piece_id=ANBM4PEUW4YPPXUYUB2VAKXMDBEXBD7CO42CXEHAJ2W5NDNXL4WQ, seg_pos={0 0}, upload=145547]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 5.161.89.252:10314->81.171.14.70:4029: use of closed network connection; write tcp 5.161.89.252:10314->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:10314->81.171.14.70:4029: use of closed network connection
Feb 24 17:35:56.827285327 (165h27m15.966269551s) [seg_pos={0 0}, upload=145547]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Feb 24 17:35:56.937070410 (165h27m16.076054634s) [upload=145547]: Done uploading segments...
Feb 24 17:35:56.937143686 (165h27m16.076127911s) [upload=145547]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 2.179432724s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
30.000861852s trace:
https://jaeger.dev.storj.io/trace/2b47b96b8b0b627e
FAIL (2026-02-24 18:15:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000873474s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
30.000767866s trace:
https://jaeger.dev.storj.io/trace/2ab96707d4885de8
FAIL (2026-02-24 18:15:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.000787734s
edge
gateway
us-select-1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
25.434728ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Length=274 Content-Type=application/xml Server=Storj Vary=Origin X-Xss-Protection=1; mode=block Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=1897417527A21368 X-Request-Id=XNNm55EmB63 Date=Tue, 24 Feb 2026 18:15:49 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 trace:
https://jaeger.dev.storj.io/trace/511603683a1407a1
FAIL (2026-02-24 18:15:49 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897417527A21368</RequestId><HostId></HostId></Error>"
Duration: 25.457882ms
edge
gateway
vhost
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
1m0.001539177s trace:
https://jaeger.dev.storj.io/trace/119f2bd0f03eceb4
FAIL (2026-02-24 18:15:46 UTC)
operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001557862s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
30.002126434s trace:
https://jaeger.dev.storj.io/trace/21e2b65806e8c0bd
FAIL (2026-02-24 18:15:45 UTC)
operation: sync2.Copy uplink: encryption: context deadline exceeded
Duration: 30.002144388s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
1m0.001834529s trace:
https://jaeger.dev.storj.io/trace/5a9e2e3249c37545
FAIL (2026-02-24 18:15:44 UTC)
operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 24 18:14:44.035429420 (166h6m3.174413635s) [upload=146121]: Starting upload Feb 24 18:14:44.035521664 (166h6m3.174505877s) [upload=146121]: Done starting upload Feb 24 18:14:44.035556908 (166h6m3.174541133s) [upload=146121]: Uploading segments... Feb 24 18:14:44.035888959 (166h6m3.174873174s) [upload=146121]: Got next segment. Inline: false Feb 24 18:14:44.035970061 (166h6m3.174954275s) [upload=146121]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:14:44.035976533 (166h6m3.174960748s) [upload=146121]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 24 18:15:44.020946093 (166h7m3.159930308s) [upload=146121]: Next segment err: context deadline exceeded Feb 24 18:15:44.021288384 (166h7m3.160272607s) [upload=146121]: Done uploading segments... Feb 24 18:15:44.021338617 (166h7m3.160322832s) [upload=146121]: Upload finished. err: context deadline exceeded
Duration: 1m0.001848986s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.ap1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
23.023824ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Security-Policy=block-all-mixed-content Server=Storj X-Amz-Request-Id=1897417407980DB2 X-Xss-Protection=1; mode=block Content-Length=274 Content-Type=application/xml Vary=Origin X-Request-Id=5fwpSp9Ve9B Date=Tue, 24 Feb 2026 18:15:44 GMT target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwlk53gfhsby563unjoqoivp56jq&Signature=Q7FkmqtUH%2BgeU7PdNdPIPgOXqhY%3D&Expires=1771549011 trace:
https://jaeger.dev.storj.io/trace/6beab8dababcf1ef
FAIL (2026-02-24 18:15:44 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>1897417407980DB2</RequestId><HostId></HostId></Error>"
Duration: 23.040106ms
edge
gateway
vhost
bgp
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
30.001233005s trace:
https://jaeger.dev.storj.io/trace/2f4d1111473d35e4
FAIL (2026-02-24 18:15:42 UTC)
operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== Feb 24 18:15:12.306373823 (166h6m31.445358017s) [upload=146128]: Starting upload Feb 24 18:15:12.306429587 (166h6m31.445413772s) [upload=146128]: Done starting upload Feb 24 18:15:12.306494269 (166h6m31.445478463s) [upload=146128]: Uploading segments... Feb 24 18:15:12.306666880 (166h6m31.445651074s) [upload=146128]: Got next segment. Inline: false Feb 24 18:15:12.306696686 (166h6m31.44568087s) [upload=146128]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:15:12.306700543 (166h6m31.445684737s) [upload=146128]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Feb 24 18:15:42.291040216 (166h7m1.43002443s) [upload=146128]: Next segment err: context deadline exceeded Feb 24 18:15:42.291407612 (166h7m1.430391827s) [upload=146128]: Done uploading segments... Feb 24 18:15:42.291460982 (166h7m1.430445197s) [upload=146128]: Upload finished. err: context deadline exceeded
Duration: 30.001249785s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
1m0.001470189s trace:
https://jaeger.dev.storj.io/trace/6ebdb6482a422500
FAIL (2026-02-24 18:15:38 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Feb 24 18:14:38.770955812 (166h5m57.909940006s) [upload=146120]: Starting upload Feb 24 18:14:38.770995686 (166h5m57.909979871s) [upload=146120]: Done starting upload Feb 24 18:14:38.771038887 (166h5m57.910023082s) [upload=146120]: Uploading segments... Feb 24 18:14:38.771067139 (166h5m57.910051324s) [upload=146120]: Got next segment. Inline: true Feb 24 18:14:38.771070666 (166h5m57.91005486s) [upload=146120]: Waiting for error group managing segments... Feb 24 18:14:38.771111373 (166h5m57.910095567s) [upload=146120]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:14:38.771115069 (166h5m57.910099263s) [upload=146120]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Feb 24 18:14:38.771117544 (166h5m57.910101738s) [upload=146120]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Feb 24 18:15:38.758100086 (166h6m57.897084271s) [upload=146120]: Done uploading segments... Feb 24 18:15:38.758139129 (166h6m57.897123313s) [upload=146120]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.001479897s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.eu1.gateway)antiblip:
60 out of 60 is failed (threshold=3) [XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX] duration:
39.497761ms remote:
"tcp:136.0.77.3:443" response headers:
Accept-Ranges=bytes Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Date=Tue, 24 Feb 2026 18:15:37 GMT Content-Length=274 Content-Security-Policy=block-all-mixed-content Vary=Origin X-Amz-Request-Id=189741725331843F X-Request-Id=REKiHNagewu target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jwhnbh22a7huxgm5mpnu45kaahuq&Signature=gznnRNzM%2FpTrDIdAeC%2Bpz3LKg0A%3D&Expires=1771549318 trace:
https://jaeger.dev.storj.io/trace/17b5d375c76a59d1
FAIL (2026-02-24 18:15:37 UTC)
orbiter warning: unexpected status code "403 Forbidden" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>AccessDenied</Code><Message>Request has expired</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>189741725331843F</RequestId><HostId></HostId></Error>"
Duration: 39.526967ms
edge
gateway
vhost
bgp
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
30.001862043s trace:
https://jaeger.dev.storj.io/trace/720f8308d7740f51
FAIL (2026-02-24 18:15:35 UTC)
operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Feb 24 18:15:05.387736563 (166h6m24.526720777s) [upload=146126]: Starting upload Feb 24 18:15:05.387827704 (166h6m24.526811918s) [upload=146126]: Done starting upload Feb 24 18:15:05.387975419 (166h6m24.526959634s) [upload=146126]: Uploading segments... Feb 24 18:15:05.388037946 (166h6m24.52702216s) [upload=146126]: Got next segment. Inline: true Feb 24 18:15:05.388044317 (166h6m24.527028532s) [upload=146126]: Waiting for error group managing segments... Feb 24 18:15:05.388094852 (166h6m24.527079077s) [upload=146126]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Feb 24 18:15:05.388101365 (166h6m24.527085579s) [upload=146126]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Feb 24 18:15:05.388105222 (166h6m24.527089436s) [upload=146126]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Feb 24 18:15:35.369529912 (166h6m54.508514137s) [upload=146126]: Done uploading segments... Feb 24 18:15:35.369710149 (166h6m54.508694353s) [upload=146126]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.001883643s
core
uplink
us-select-1
|