Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 119387 reports (Page 1322 of 2388)
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
7 out of 60 is failed (threshold=3) [.X.........XX.........X.XX.X................................]
duration:
62.092365ms
remote:
"tcp:136.0.77.2:443"
response headers:
Content-Length=423
Content-Security-Policy=block-all-mixed-content
X-Amz-Request-Id=185F3AFB7EA3A7A1
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 05:47:11 GMT
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Request-Id=8tjmSuZMkvh
Accept-Ranges=bytes
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 06:11:01 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>"
Duration: 62.116059ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
17 out of 60 is failed (threshold=3) [..X...X......XX.X.XX....XX.XX.........X.XX.X.X..X...........]
duration:
525.3558ms
FAIL (2025-08-26 06:10:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 525.373814ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
5 out of 60 is failed (threshold=3) [.X......................X.....X..X........X.................]
duration:
403.443594ms
FAIL (2025-08-26 06:10:51 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
4528s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment...
Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment.
Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049     , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995     , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457    , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868     , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682     , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344    , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247  , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043     , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446    , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830     , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149    , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343  , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835     , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310     , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222    , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624     , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035     , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616  , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment
Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments...
Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222    , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049     , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995     , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043     , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310     , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624     , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830     , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149    , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682     , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344    , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835     , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457    , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035     , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446    , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces.
Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868     , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF
Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616  , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247  , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343  , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments...
Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 403.47386ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
7 out of 60 is failed (threshold=3) [............X...XX........X.............X..XX...............]
duration:
35.673606ms
remote:
"tcp:45.38.24.2:443"
response headers:
Vary=Origin
X-Amz-Request-Id=185F3B9472B13A12
X-Request-Id=GCZRZMJ5aWo
Date=Tue, 26 Aug 2025 05:58:08 GMT
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Server=Storj
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
Content-Length=433
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:10:48 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>"
Duration: 35.697241ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
19 out of 60 is failed (threshold=3) [XXXX.....XX..X.XXX......XX.X........XX.XX.XX................]
duration:
1.929446331s
FAIL (2025-08-26 06:10:47 UTC)
operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.929473542s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
12 out of 60 is failed (threshold=3) [..X......X..XX.X.......XX..........XXX....X..X..............]
duration:
1.736649612s
FAIL (2025-08-26 06:10:44 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload
Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload
Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments...
Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false
Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment...
Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment.
Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment
Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments...
Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments...
Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.736667435s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
12 out of 60 is failed (threshold=3) [..XX.XXX..........X..XX..................XX.X.X.............]
duration:
222.595909ms
FAIL (2025-08-26 06:10:42 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload
Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload
Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments...
Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true
Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments...
Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments...
Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 222.627539ms core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
2 out of 60 is failed (threshold=6) [................X..........................X................]
duration:
38.81993ms
remote:
"tcp:136.0.77.2:443"
response headers:
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Date=Tue, 26 Aug 2025 05:58:10 GMT
Accept-Ranges=bytes
X-Request-Id=2dEEZwg2VX7
Content-Type=text/html; charset=utf-8
Access-Control-Allow-Headers=*
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:10:42 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 38.843976ms edge linksharing bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
11 out of 60 is failed (threshold=3) [..XX..XX....X............XXX..............XX.X..............]
duration:
301.801032ms
FAIL (2025-08-26 06:10:42 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 301.825749ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
20 out of 60 is failed (threshold=3) [XXXXX.....XX..X.XXX......XX.X........XX.XX.XX...............]
duration:
1.702885466s
FAIL (2025-08-26 06:10:30 UTC)
operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.702904223s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
15 out of 60 is failed (threshold=3) [.......XX.....XXXXX.....XX.............X.X.X.XXX............]
duration:
186.723219ms
FAIL (2025-08-26 06:10:28 UTC)
operation: DownloadObject
uplink: metaclient: context deadline exceeded
Duration: 186.75076ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
4 out of 60 is failed (threshold=3) [X...............XX........X.................................]
duration:
622.097977ms
FAIL (2025-08-26 06:10:27 UTC)
orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)
Duration: 622.183429ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)
antiblip:
4 out of 60 is failed (threshold=3) [...............X.........XX...................X.............]
duration:
267.567813ms
FAIL (2025-08-26 06:10:25 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload
Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload
Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments...
Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true
Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments...
Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments...
Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 267.58734ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
7 out of 60 is failed (threshold=3) [..X.........XX.........X.XX.X...............................]
duration:
65.068539ms
remote:
"tcp:136.0.77.2:443"
response headers:
Content-Length=423
Content-Security-Policy=block-all-mixed-content
X-Amz-Request-Id=185F3AFB7EA3A7A1
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 05:47:11 GMT
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Request-Id=8tjmSuZMkvh
Accept-Ranges=bytes
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 06:10:16 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>"
Duration: 65.093467ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
5 out of 60 is failed (threshold=3) [..X......................X.....X..X........X................]
duration:
1.125176782s
FAIL (2025-08-26 06:10:02 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
4528s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment...
Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment.
Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049     , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995     , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457    , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868     , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682     , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344    , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247  , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043     , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446    , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830     , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149    , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343  , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835     , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310     , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222    , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624     , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035     , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616  , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment
Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments...
Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222    , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049     , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995     , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043     , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310     , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624     , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830     , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149    , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682     , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344    , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835     , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457    , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035     , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446    , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces.
Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868     , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF
Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616  , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247  , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343  , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments...
Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 1.125208242s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
2 out of 60 is failed (threshold=6) [.................X..........................X...............]
duration:
41.591306ms
remote:
"tcp:136.0.77.2:443"
response headers:
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Date=Tue, 26 Aug 2025 05:58:10 GMT
Accept-Ranges=bytes
X-Request-Id=2dEEZwg2VX7
Content-Type=text/html; charset=utf-8
Access-Control-Allow-Headers=*
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:10:02 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 41.630592ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
7 out of 60 is failed (threshold=3) [.............X...XX........X.............X..XX..............]
duration:
42.053979ms
remote:
"tcp:45.38.24.2:443"
response headers:
Vary=Origin
X-Amz-Request-Id=185F3B9472B13A12
X-Request-Id=GCZRZMJ5aWo
Date=Tue, 26 Aug 2025 05:58:08 GMT
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Server=Storj
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
Content-Length=433
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:10:01 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>"
Duration: 42.073526ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
8 out of 60 is failed (threshold=3) [X..X.........XX.........X.XX.X..............................]
duration:
208.182574ms
remote:
"tcp:136.0.77.2:443"
response headers:
Content-Length=423
Content-Security-Policy=block-all-mixed-content
X-Amz-Request-Id=185F3AFB7EA3A7A1
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 05:47:11 GMT
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Request-Id=8tjmSuZMkvh
Accept-Ranges=bytes
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 06:09:55 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>"
Duration: 208.244201ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)
antiblip:
4 out of 60 is failed (threshold=3) [................X.........XX...................X............]
duration:
209.192852ms
FAIL (2025-08-26 06:09:49 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload
Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload
Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments...
Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true
Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments...
Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments...
Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 209.217178ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
12 out of 60 is failed (threshold=3) [...X......X..XX.X.......XX..........XXX....X..X.............]
duration:
2.089821716s
FAIL (2025-08-26 06:09:47 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload
Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload
Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments...
Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false
Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment...
Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment.
Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment
Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments...
Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments...
Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.089841353s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
21 out of 60 is failed (threshold=3) [XXXXXX.....XX..X.XXX......XX.X........XX.XX.XX..............]
duration:
1.805028864s
FAIL (2025-08-26 06:09:47 UTC)
operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.805049072s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
17 out of 60 is failed (threshold=3) [...X...X......XX.X.XX....XX.XX.........X.XX.X.X..X..........]
duration:
516.423393ms
FAIL (2025-08-26 06:09:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 516.439413ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
11 out of 60 is failed (threshold=3) [...XX..XX....X............XXX..............XX.X.............]
duration:
441.335183ms
FAIL (2025-08-26 06:09:45 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 441.351693ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
5 out of 60 is failed (threshold=3) [XX...............XX........X................................]
duration:
591.787128ms
FAIL (2025-08-26 06:09:38 UTC)
orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)
Duration: 591.808497ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
2 out of 60 is failed (threshold=6) [..................X..........................X..............]
duration:
50.365026ms
remote:
"tcp:136.0.77.2:443"
response headers:
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Date=Tue, 26 Aug 2025 05:58:10 GMT
Accept-Ranges=bytes
X-Request-Id=2dEEZwg2VX7
Content-Type=text/html; charset=utf-8
Access-Control-Allow-Headers=*
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:09:37 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 50.394151ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
16 out of 60 is failed (threshold=3) [X.......XX.....XXXXX.....XX.............X.X.X.XXX...........]
duration:
388.621768ms
FAIL (2025-08-26 06:09:32 UTC)
operation: DownloadObject
uplink: metaclient: context deadline exceeded
Duration: 388.649279ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
12 out of 60 is failed (threshold=3) [...XX.XXX..........X..XX..................XX.X.X............]
duration:
148.196282ms
FAIL (2025-08-26 06:09:30 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload
Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload
Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments...
Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true
Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments...
Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments...
Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 148.218984ms core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
7 out of 60 is failed (threshold=3) [..............X...XX........X.............X..XX.............]
duration:
41.506418ms
remote:
"tcp:45.38.24.2:443"
response headers:
Vary=Origin
X-Amz-Request-Id=185F3B9472B13A12
X-Request-Id=GCZRZMJ5aWo
Date=Tue, 26 Aug 2025 05:58:08 GMT
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Server=Storj
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
Content-Length=433
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:09:30 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>"
Duration: 41.520173ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
13 out of 60 is failed (threshold=3) [X...X......X..XX.X.......XX..........XXX....X..X............]
duration:
2.00353939s
FAIL (2025-08-26 06:09:28 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload
Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload
Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments...
Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false
Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment...
Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment.
Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment
Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments...
Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments...
Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded
Duration: 2.003563336s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
11 out of 60 is failed (threshold=3) [....XX..XX....X............XXX..............XX.X............]
duration:
203.52585ms
FAIL (2025-08-26 06:09:17 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 203.554934ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
5 out of 60 is failed (threshold=3) [.XX...............XX........X...............................]
duration:
669.820579ms
FAIL (2025-08-26 06:09:16 UTC)
orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)
Duration: 669.839807ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
17 out of 60 is failed (threshold=3) [....X...X......XX.X.XX....XX.XX.........X.XX.X.X..X.........]
duration:
609.67045ms
FAIL (2025-08-26 06:09:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 609.685228ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
16 out of 60 is failed (threshold=3) [.X.......XX.....XXXXX.....XX.............X.X.X.XXX..........]
duration:
197.356783ms
FAIL (2025-08-26 06:08:56 UTC)
operation: DownloadObject
uplink: metaclient: context deadline exceeded
Duration: 197.385147ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)
antiblip:
4 out of 60 is failed (threshold=3) [.................X.........XX...................X...........]
duration:
268.956802ms
FAIL (2025-08-26 06:08:55 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload
Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload
Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments...
Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true
Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments...
Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments...
Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 268.979394ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
6 out of 60 is failed (threshold=3) [X..X......................X.....X..X........X...............]
duration:
2.68828054s
FAIL (2025-08-26 06:08:50 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
4528s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment...
Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment.
Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049     , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995     , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457    , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868     , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682     , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344    , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247  , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043     , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446    , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830     , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149    , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343  , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835     , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310     , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222    , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624     , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035     , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616  , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment
Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments...
Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222    , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049     , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995     , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043     , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310     , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624     , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830     , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149    , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682     , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344    , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835     , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457    , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035     , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446    , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces.
Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868     , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF
Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616  , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247  , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343  , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments...
Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 2.688326006s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
13 out of 60 is failed (threshold=3) [.X...X......X..XX.X.......XX..........XXX....X..X...........]
duration:
3.324676703s
FAIL (2025-08-26 06:08:45 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload
Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload
Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments...
Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false
Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment...
Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment.
Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment
Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments...
Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments...
Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.324698544s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
8 out of 60 is failed (threshold=3) [.X..X.........XX.........X.XX.X.............................]
duration:
68.612022ms
remote:
"tcp:136.0.77.2:443"
response headers:
Content-Length=423
Content-Security-Policy=block-all-mixed-content
X-Amz-Request-Id=185F3AFB7EA3A7A1
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 05:47:11 GMT
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Request-Id=8tjmSuZMkvh
Accept-Ranges=bytes
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 06:08:44 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>"
Duration: 68.629787ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
22 out of 60 is failed (threshold=3) [XXXXXXX.....XX..X.XXX......XX.X........XX.XX.XX.............]
duration:
1.86706045s
FAIL (2025-08-26 06:08:41 UTC)
operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.867079899s core uplink us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
2 out of 60 is failed (threshold=6) [...................X..........................X.............]
duration:
43.812216ms
remote:
"tcp:136.0.77.2:443"
response headers:
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Date=Tue, 26 Aug 2025 05:58:10 GMT
Accept-Ranges=bytes
X-Request-Id=2dEEZwg2VX7
Content-Type=text/html; charset=utf-8
Access-Control-Allow-Headers=*
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 06:08:39 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 43.835441ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)
antiblip:
7 out of 60 is failed (threshold=3) [...............X...XX........X.............X..XX............]
duration:
26.201411ms
remote:
"tcp:45.38.24.2:443"
response headers:
Vary=Origin
X-Amz-Request-Id=185F3B9472B13A12
X-Request-Id=GCZRZMJ5aWo
Date=Tue, 26 Aug 2025 05:58:08 GMT
Content-Security-Policy=block-all-mixed-content
Content-Type=application/xml
Server=Storj
X-Xss-Protection=1; mode=block
Accept-Ranges=bytes
Content-Length=433
target:
https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410
FAIL (2025-08-26 06:08:37 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>"
Duration: 26.219705ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
5 out of 60 is failed (threshold=3) [..XX...............XX........X..............................]
duration:
578.81428ms
FAIL (2025-08-26 06:08:36 UTC)
orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)
Duration: 578.830471ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
6 out of 60 is failed (threshold=3) [.X..X......................X.....X..X........X..............]
duration:
1.446171073s
FAIL (2025-08-26 06:08:31 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
4528s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment...
Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment.
Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049     , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995     , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457    , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868     , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682     , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344    , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247  , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043     , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446    , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830     , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149    , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343  , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835     , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310     , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222    , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624     , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035     , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616  , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece...
Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment
Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments...
Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222    , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049     , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995     , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043     , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310     , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624     , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830     , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149    , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682     , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344    , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835     , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457    , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035     , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446    , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil>
Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces.
Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868     , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF
Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616  , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247  , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343  , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments...
Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 1.446189608s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
8 out of 60 is failed (threshold=3) [..X..X.........XX.........X.XX.X............................]
duration:
41.342077ms
remote:
"tcp:136.0.77.2:443"
response headers:
Content-Length=423
Content-Security-Policy=block-all-mixed-content
X-Amz-Request-Id=185F3AFB7EA3A7A1
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 05:47:11 GMT
Content-Type=application/xml
Server=Storj
Vary=Origin
X-Request-Id=8tjmSuZMkvh
Accept-Ranges=bytes
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 06:08:28 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>"
Duration: 41.3651ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
13 out of 60 is failed (threshold=3) [X...XX.XXX..........X..XX..................XX.X.X...........]
duration:
159.130746ms
FAIL (2025-08-26 06:08:27 UTC)
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload
Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload
Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments...
Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true
Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments...
Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments...
Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 159.151585ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
22 out of 60 is failed (threshold=3) [.XXXXXXX.....XX..X.XXX......XX.X........XX.XX.XX............]
duration:
1.679256888s
FAIL (2025-08-26 06:08:19 UTC)
operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 1.679283187s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
11 out of 60 is failed (threshold=3) [.....XX..XX....X............XXX..............XX.X...........]
duration:
484.974546ms
FAIL (2025-08-26 06:08:18 UTC)
orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 485.022986ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
16 out of 60 is failed (threshold=3) [..X.......XX.....XXXXX.....XX.............X.X.X.XXX.........]
duration:
791.110031ms
FAIL (2025-08-26 06:08:08 UTC)
operation: DownloadObject
uplink: metaclient: context deadline exceeded
Duration: 791.129999ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
18 out of 60 is failed (threshold=3) [X....X...X......XX.X.XX....XX.XX.........X.XX.X.X..X........]
duration:
390.584824ms
FAIL (2025-08-26 06:07:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 390.606724ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
5 out of 60 is failed (threshold=3) [...XX...............XX........X.............................]
duration:
767.501821ms
FAIL (2025-08-26 06:07:51 UTC)
orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)
Duration: 767.52254ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
13 out of 60 is failed (threshold=3) [..X...X......X..XX.X.......XX..........XXX....X..X..........]
duration:
449.601596ms
FAIL (2025-08-26 06:07:47 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload
Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload
Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments...
Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false
Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment...
Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment.
Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece...
Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment
Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments...
Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975     , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298     , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735     , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879     , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843     , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225    , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190    , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589     , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807     , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553     , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115    , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909     , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519     , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil>
Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces.
Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083    , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113    , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007     , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF
Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments...
Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded
Duration: 449.635068ms core uplink us-select-1