Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.XX........X................................................] duration: 643.032755ms FAIL (2025-08-26 06:20:03 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 643.059425ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 6 out of 60 is failed (threshold=3) [.XX........X.............X..XX..............................] duration: 44.588702ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:20:01 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 44.614582ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [.....X..........................X...........................] duration: 97.854185ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:58:10 GMT Accept-Ranges=bytes X-Request-Id=2dEEZwg2VX7 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 06:20:00 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 97.876788ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [.XXXXX.....XX.............X.X.X.XXX.........................] duration: 248.197617ms FAIL (2025-08-26 06:19:55 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 248.242591ms core uplink us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 5 out of 60 is failed (threshold=3) [X.........X.XX.X............................................] duration: 42.433722ms remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 06:19:46 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 42.461113ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [......X..........................X..........................] duration: 40.135621ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:58:10 GMT Accept-Ranges=bytes X-Request-Id=2dEEZwg2VX7 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 06:19:39 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 40.156911ms edge linksharing bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.XX.X.XX....XX.XX.........X.XX.X.X..X.......................] duration: 372.760983ms FAIL (2025-08-26 06:19:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 372.772465ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.X............XXX..............XX.X.........................] duration: 212.649917ms FAIL (2025-08-26 06:19:28 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 212.85327ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [..XXXXX.....XX.............X.X.X.XXX........................] duration: 240.308668ms FAIL (2025-08-26 06:19:20 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 240.340568ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XX.X.......XX..........XXX....X..X..........................] duration: 2.199779041s FAIL (2025-08-26 06:19:16 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments... Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment... Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment. Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments... Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces. Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments... Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.199799729s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..XX........X...............................................] duration: 836.36641ms FAIL (2025-08-26 06:19:13 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 836.379856ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [.X.........XX...................X...........................] duration: 176.274763ms FAIL (2025-08-26 06:19:10 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments... Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments... Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments... Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 176.297965ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [..XX.X.XX....XX.XX.........X.XX.X.X..X......................] duration: 565.328319ms FAIL (2025-08-26 06:19:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 565.353836ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 13 out of 60 is failed (threshold=3) [X.XXX......XX.X........XX.XX.XX.............................] duration: 2.25263364s FAIL (2025-08-26 06:19:06 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.252655562s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X.....X..X........X...........................] duration: 694.051163ms FAIL (2025-08-26 06:19:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 694.081028ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 6 out of 60 is failed (threshold=3) [..XX........X.............X..XX.............................] duration: 17.66703ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:19:03 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 17.684384ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [......X..XX..................XX.X.X.........................] duration: 475.648383ms FAIL (2025-08-26 06:18:58 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments... Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments... Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments... Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 475.664964ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.XX.X.......XX..........XXX....X..X.........................] duration: 2.042760569s FAIL (2025-08-26 06:18:51 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments... Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment... Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment. Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments... Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces. Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments... Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.042784224s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [..X.........XX...................X..........................] duration: 150.730737ms FAIL (2025-08-26 06:18:47 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments... Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments... Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments... Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 150.761706ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.X.XXX......XX.X........XX.XX.XX............................] duration: 1.707552096s FAIL (2025-08-26 06:18:41 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.707579608s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X.....X..X........X..........................] duration: 1.866558229s FAIL (2025-08-26 06:18:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 1.866579811s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 6 out of 60 is failed (threshold=3) [XX.........X.XX.X...........................................] duration: 62.903567ms remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 06:18:40 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 62.92166ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [..X............XXX..............XX.X........................] duration: 175.843991ms FAIL (2025-08-26 06:18:38 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 175.861114ms edge gateway us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [.......X..........................X.........................] duration: 38.274854ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:58:10 GMT Accept-Ranges=bytes X-Request-Id=2dEEZwg2VX7 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 06:18:37 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 38.292047ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [...XXXXX.....XX.............X.X.X.XXX.......................] duration: 214.959852ms FAIL (2025-08-26 06:18:35 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 214.989818ms core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 6 out of 60 is failed (threshold=3) [...XX........X.............X..XX............................] duration: 26.875924ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:18:33 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 26.901693ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [...XX.X.XX....XX.XX.........X.XX.X.X..X.....................] duration: 488.363584ms FAIL (2025-08-26 06:18:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 488.383731ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [...X.........XX...................X.........................] duration: 133.941651ms FAIL (2025-08-26 06:18:25 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments... Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments... Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments... Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 133.964413ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.......X..XX..................XX.X.X........................] duration: 171.664155ms FAIL (2025-08-26 06:18:16 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments... Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments... Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments... Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 171.687789ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 13 out of 60 is failed (threshold=3) [..X.XXX......XX.X........XX.XX.XX...........................] duration: 1.73083929s FAIL (2025-08-26 06:18:13 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.730862743s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...XX........X..............................................] duration: 804.62ms FAIL (2025-08-26 06:18:10 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 804.834594ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [....XX.X.XX....XX.XX.........X.XX.X.X..X....................] duration: 419.701386ms FAIL (2025-08-26 06:18:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 419.734097ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [........X..XX..................XX.X.X.......................] duration: 165.264014ms FAIL (2025-08-26 06:17:54 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments... Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments... Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments... Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 165.285905ms core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 7 out of 60 is failed (threshold=3) [X...XX........X.............X..XX...........................] duration: 22.422674ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:17:52 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 22.439686ms edge gateway vhost us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 6 out of 60 is failed (threshold=3) [.XX.........X.XX.X..........................................] duration: 40.515267ms remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 06:17:51 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 40.562688ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....XX.X.XX....XX.XX.........X.XX.X.X..X...................] duration: 459.889057ms FAIL (2025-08-26 06:17:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 459.911258ms edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....XX........X.............................................] duration: 615.018092ms FAIL (2025-08-26 06:17:46 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 615.083233ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..XX.X.......XX..........XXX....X..X........................] duration: 3.370957089s FAIL (2025-08-26 06:17:44 UTC) operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments... Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment... Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment. Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments... Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces. Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments... Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.37098299s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [........X..........................X........................] duration: 48.793747ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:58:10 GMT Accept-Ranges=bytes X-Request-Id=2dEEZwg2VX7 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 06:17:44 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 48.818804ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 4 out of 60 is failed (threshold=3) [....X.........XX...................X........................] duration: 167.331208ms FAIL (2025-08-26 06:17:39 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments... Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments... Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments... Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 167.354101ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 7 out of 60 is failed (threshold=3) [.........X..XX..................XX.X.X......................] duration: 166.14781ms FAIL (2025-08-26 06:17:38 UTC) operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments... Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments... Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments... Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 166.175222ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 14 out of 60 is failed (threshold=3) [X..X.XXX......XX.X........XX.XX.XX..........................] duration: 2.807051727s FAIL (2025-08-26 06:17:36 UTC) operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.80707427s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 13 out of 60 is failed (threshold=3) [....XXXXX.....XX.............X.X.X.XXX......................] duration: 167.998245ms FAIL (2025-08-26 06:17:36 UTC) operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 168.011821ms core uplink us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 7 out of 60 is failed (threshold=3) [.X...XX........X.............X..XX..........................] duration: 68.381167ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:17:36 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 68.405224ms edge gateway vhost us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....XX........X............................................] duration: 682.172058ms FAIL (2025-08-26 06:17:30 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 682.251606ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X.....X..X........X.........................] duration: 482.777557ms FAIL (2025-08-26 06:17:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 482.86385ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [...X............XXX..............XX.X.......................] duration: 235.804851ms FAIL (2025-08-26 06:17:25 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 235.828095ms edge gateway us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 2 out of 60 is failed (threshold=6) [.........X..........................X.......................] duration: 226.008613ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:58:10 GMT Accept-Ranges=bytes X-Request-Id=2dEEZwg2VX7 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 06:17:23 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 226.03344ms edge linksharing bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......XX........X...........................................] duration: 869.375044ms FAIL (2025-08-26 06:17:15 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 869.391363ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 7 out of 60 is failed (threshold=3) [..X...XX........X.............X..XX.........................] duration: 19.922882ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:17:10 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 19.950394ms edge gateway vhost us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)