Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [............X.X.X.XXX.......................................] duration: 229.808047ms FAIL (2025-08-26 06:29:28 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 229.831021ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.........XXX....X..X........................................] duration: 3.132533069s FAIL (2025-08-26 06:29:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments... Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment... Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment. Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments... Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces. Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments... Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.132633147s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [XXX..............XX.X.......................................] duration: 182.685459ms FAIL (2025-08-26 06:29:18 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 182.704856ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 56.504114ms remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 06:29:14 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 56.521777ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [XX.X........XX.XX.XX........................................] duration: 1.89821213s FAIL (2025-08-26 06:29:13 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.898233601s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [XX.XX.........X.XX.X.X..X...................................] duration: 411.010384ms FAIL (2025-08-26 06:29:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 411.299228ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 331.10775ms FAIL (2025-08-26 06:29:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments... Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments... Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments... Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 331.128741ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [............X..XX...........................................] duration: 24.872365ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:29:05 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 24.893365ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X.....X..X........X......................................] duration: 1.534038247s FAIL (2025-08-26 06:28:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 1.534063084s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 4 out of 60 is failed (threshold=3) [................XX.X.X......................................] duration: 345.723197ms FAIL (2025-08-26 06:28:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments... Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments... Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments... Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 345.761459ms core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [......................X.....................................] duration: 98.949675ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:58:10 GMT Accept-Ranges=bytes X-Request-Id=2dEEZwg2VX7 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 06:28:51 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 98.980363ms edge linksharing bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.XXX..............XX.X......................................] duration: 176.456483ms FAIL (2025-08-26 06:28:48 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 176.474108ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 2 out of 60 is failed (threshold=3) [X...................X.......................................] duration: 162.508961ms FAIL (2025-08-26 06:28:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments... Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments... Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments... Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 162.526183ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X.....X..X........X.....................................] duration: 376.659061ms FAIL (2025-08-26 06:28:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 376.685351ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 44.858818ms remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 06:28:30 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 44.889416ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 4 out of 60 is failed (threshold=3) [.................XX.X.X.....................................] duration: 161.20868ms FAIL (2025-08-26 06:28:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments... Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments... Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments... Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 161.230262ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.XX.X........XX.XX.XX.......................................] duration: 2.801156696s FAIL (2025-08-26 06:28:28 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.801201398s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [.............X.X.X.XXX......................................] duration: 180.539836ms FAIL (2025-08-26 06:28:24 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 180.560034ms core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 962.859723ms FAIL (2025-08-26 06:28:23 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 962.882196ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..........XXX....X..X.......................................] duration: 537.062477ms FAIL (2025-08-26 06:28:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments... Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment... Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment. Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments... Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces. Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments... Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded Duration: 537.09008ms core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.......................X....................................] duration: 46.63866ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:58:10 GMT Accept-Ranges=bytes X-Request-Id=2dEEZwg2VX7 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 06:28:20 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 46.663546ms edge linksharing bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.XX.XX.........X.XX.X.X..X..................................] duration: 455.114912ms FAIL (2025-08-26 06:28:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 455.132736ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [.............X..XX..........................................] duration: 28.646605ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:28:14 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 28.663868ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [X..........XXX....X..X......................................] duration: 3.778531631s FAIL (2025-08-26 06:28:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments... Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment... Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment. Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments... Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces. Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments... Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.778556628s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 4 out of 60 is failed (threshold=3) [..................XX.X.X....................................] duration: 129.515675ms FAIL (2025-08-26 06:27:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments... Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments... Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments... Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 129.535263ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 3 out of 60 is failed (threshold=3) [XX...................X......................................] duration: 142.178388ms FAIL (2025-08-26 06:27:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments... Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments... Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments... Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 142.197324ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [..XXX..............XX.X.....................................] duration: 190.501512ms FAIL (2025-08-26 06:27:42 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 190.524415ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..XX.XX.........X.XX.X.X..X.................................] duration: 460.444787ms FAIL (2025-08-26 06:27:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 460.471426ms edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [XX.X........................................................] duration: 46.264925ms remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 06:27:39 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 46.292358ms edge gateway vhost bgp us1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [X.............X..XX.........................................] duration: 20.062956ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:27:35 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 20.081822ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..XX.X........XX.XX.XX......................................] duration: 1.746689485s FAIL (2025-08-26 06:27:32 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.746722969s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XX..........XXX....X..X.....................................] duration: 2.645856426s FAIL (2025-08-26 06:27:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Aug 26 06:01:16.935075318 (13h57m32.160258395s) [upload=12249]: Starting upload Aug 26 06:01:16.935159125 (13h57m32.160342203s) [upload=12249]: Done starting upload Aug 26 06:01:16.935236191 (13h57m32.160419248s) [upload=12249]: Uploading segments... Aug 26 06:01:16.935514284 (13h57m32.160697362s) [upload=12249]: Got next segment. Inline: false Aug 26 06:01:16.935553739 (13h57m32.160736815s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:01:16.935559128 (13h57m32.160742206s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 06:01:20.960309229 (13h57m36.185492306s) [seg_pos={0 0}, upload=12249]: Begin upload segment... Aug 26 06:01:20.960355064 (13h57m36.185538142s) [seg_pos={0 0}, upload=12249]: Done begin upload segment. Aug 26 06:01:20.960387716 (13h57m36.185570793s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960408445 (13h57m36.185591522s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960886374 (13h57m36.186069452s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.960910280 (13h57m36.186093357s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961258585 (13h57m36.186441662s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961347041 (13h57m36.186530119s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961690488 (13h57m36.186873566s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.961853976 (13h57m36.187037093s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962053672 (13h57m36.187236748s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962279637 (13h57m36.187462715s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962399853 (13h57m36.18758294s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.962813241 (13h57m36.187996318s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963313903 (13h57m36.18849699s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.963784729 (13h57m36.188967816s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.964647042 (13h57m36.189830119s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:20.966378030 (13h57m36.191561107s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Uploading piece... Aug 26 06:01:21.091173803 (13h57m36.31635685s) [upload=12249]: Next returned nil segment Aug 26 06:01:21.091186106 (13h57m36.316369163s) [upload=12249]: Waiting for error group managing segments... Aug 26 06:01:21.099963365 (13h57m36.325146412s) [noise=true , address=45.91.116.6:4975 , piece_id=JMZSNJPJONSH5FQ3JCZ46NIJYDZ44VJAFF42HRJX3E3LG534J3GA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100375901 (13h57m36.325558958s) [noise=true , address=45.91.116.5:4298 , piece_id=WXKBEQTFN26RVXJOZ5ZW3YMGU2DLHA4GXTTBM66GJDGKXMS57XCA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100581919 (13h57m36.325764966s) [noise=true , address=77.90.31.16:4735 , piece_id=FDLX6N366VI2HPOZSRXP7IPUWS2W5BFRHQOOWLGNYG2SOFXRUCUQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.100961384 (13h57m36.32614443s) [noise=true , address=77.90.31.21:4879 , piece_id=CWKEHIKRLWJIH6AT4Q3NNH2BEQ7IL37LA3SMFOPZHFUSTZ2DO4BQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101547196 (13h57m36.326730243s) [noise=true , address=77.90.31.19:4843 , piece_id=QQIEGA6QZDVOCMTDHBUMGPAXFZDEYZS4FJBM5MXEOGIUT5I6IIIQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.101589245 (13h57m36.326772303s) [noise=true , address=45.91.116.12:4225 , piece_id=AGWW55QSYNIWRHTNOA33ECCW6Y73MGYNV5YINLL3AUNGFVVRKVGQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102192681 (13h57m36.327375728s) [noise=true , address=45.91.116.14:4190 , piece_id=POKZWWMHMG7QDCJAZDHXK2KVPJFG2I3TUDIVFPGS6DKUI2HITMQQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.102908088 (13h57m36.328091134s) [noise=true , address=77.90.31.12:4589 , piece_id=BPLV3L7ZQFMPNNXGRH6WFZBPULIN4Q4RTU6IT2O6H5JAPQZ4F44A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.103332385 (13h57m36.328515433s) [noise=true , address=77.90.31.18:4807 , piece_id=ROOLNX5IUH2ADQQKE5YFLBJ2BWHFDDVJOTHSEU2OXM3OTL3STX2Q, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.104387741 (13h57m36.329570798s) [noise=true , address=77.90.31.11:4553 , piece_id=EUNDLK3M2BUMVOK3GR2JMUAS2RKHXSYD4CIVH7Q57XO3PFWWQTNQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.105248913 (13h57m36.33043198s) [noise=true , address=104.36.82.11:4115 , piece_id=HQOBP24PBR2ASGXIJPBWIUZBJGX4F7QWHNW5MC6OQHQ4BVDZQ7CA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.106056983 (13h57m36.33124005s) [noise=true , address=104.36.82.7:4909 , piece_id=ZXUJWXR44KKEZ6E6TH6XR6MFUOYGQJ3YDCTHKP4LHTIOLM6RL77A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121516650 (13h57m36.346699717s) [noise=true , address=77.90.31.10:4519 , piece_id=2FK5BFR7AWFJRVJHPUTUHUS4MNKI5JHYBTRY6Y7OQP6BOVQM3PUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: <nil> Aug 26 06:01:21.121544572 (13h57m36.346727649s) [seg_pos={0 0}, upload=12249]: Segment reached optimal threshold of 13 pieces. Aug 26 06:01:21.121701718 (13h57m36.346884795s) [noise=true , address=45.91.116.10:4083 , piece_id=RIQ325WNN7S3RRNZNB3OTC4XHGC7P7XDWJE5UGJAK7OFIPZWLZUA, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121750159 (13h57m36.346933236s) [noise=true , address=104.36.82.11:4113 , piece_id=2PLIWTNSJDCR2S6DCG2IO3DDUDNVOUNYJ3WJZ4MDTOBHYRTWG3FQ, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121831041 (13h57m36.347014099s) [noise=true , address=104.36.82.9:4007 , piece_id=JINUPD7LQN6ENPKCRIPYBRKA3MZGQZHGUGOSTHQ5YQ7FUVOKP32A, seg_pos={0 0}, upload=12249]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp): protocol: expected piece hash; context canceled; EOF Aug 26 06:01:21.121871057 (13h57m36.347054124s) [seg_pos={0 0}, upload=12249]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Aug 26 06:01:21.121991063 (13h57m36.347174129s) [upload=12249]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 06:01:21.121996253 (13h57m36.34717932s) [upload=12249]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:01:46.936407433 (13h58m2.161590511s) [upload=12249]: Done uploading segments... Aug 26 06:01:46.936438953 (13h58m2.161622029s) [upload=12249]: Upload finished. err: metaclient: context deadline exceeded Duration: 2.645875361s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X.....X..X........X....................................] duration: 505.561661ms FAIL (2025-08-26 06:27:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 505.583073ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 696.832715ms FAIL (2025-08-26 06:27:30 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 696.848024ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...XX.XX.........X.XX.X.X..X................................] duration: 580.717844ms FAIL (2025-08-26 06:27:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 580.733905ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 7 out of 60 is failed (threshold=3) [X.............X.X.X.XXX.....................................] duration: 187.375525ms FAIL (2025-08-26 06:27:21 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 187.397787ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 3 out of 60 is failed (threshold=3) [.XX...................X.....................................] duration: 154.477981ms FAIL (2025-08-26 06:27:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments... Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments... Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments... Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 154.51408ms core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [........................X...................................] duration: 42.061586ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:58:10 GMT Accept-Ranges=bytes X-Request-Id=2dEEZwg2VX7 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 06:27:18 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 42.087645ms edge linksharing bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway) antiblip: 3 out of 60 is failed (threshold=3) [.XX.X.......................................................] duration: 55.062269ms remote: "tcp:136.0.77.2:443" response headers: Content-Length=423 Content-Security-Policy=block-all-mixed-content X-Amz-Request-Id=185F3AFB7EA3A7A1 X-Xss-Protection=1; mode=block Date=Tue, 26 Aug 2025 05:47:11 GMT Content-Type=application/xml Server=Storj Vary=Origin X-Request-Id=8tjmSuZMkvh Accept-Ranges=bytes target: https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222 FAIL (2025-08-26 06:27:15 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3AFB7EA3A7A1</RequestId><HostId></HostId></Error>" Duration: 55.091122ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 606.239086ms FAIL (2025-08-26 06:27:13 UTC) orbiter warning: operation error S3: GetObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 185F3AEE3B34C493, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled) Duration: 606.261268ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 5 out of 60 is failed (threshold=3) [X..................XX.X.X...................................] duration: 193.910505ms FAIL (2025-08-26 06:27:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments... Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments... Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments... Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 193.930934ms core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway) antiblip: 4 out of 60 is failed (threshold=3) [.X.............X..XX........................................] duration: 21.245143ms remote: "tcp:45.38.24.2:443" response headers: Vary=Origin X-Amz-Request-Id=185F3B9472B13A12 X-Request-Id=GCZRZMJ5aWo Date=Tue, 26 Aug 2025 05:58:08 GMT Content-Security-Policy=block-all-mixed-content Content-Type=application/xml Server=Storj X-Xss-Protection=1; mode=block Accept-Ranges=bytes Content-Length=433 target: https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410 FAIL (2025-08-26 06:27:08 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks-us-select</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3B9472B13A12</RequestId><HostId></HostId></Error>" Duration: 21.26486ms edge gateway vhost us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X.....X..X........X...................................] duration: 1.258081896s FAIL (2025-08-26 06:26:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== 4528s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 05:56:07.875566731 (13h52m23.100749778s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 26 05:56:07.989747436 (13h52m23.214930493s) [seg_pos={0 0}, upload=12168]: Begin upload segment... Aug 26 05:56:07.989793102 (13h52m23.21497616s) [seg_pos={0 0}, upload=12168]: Done begin upload segment. Aug 26 05:56:07.989813511 (13h52m23.214996558s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989825303 (13h52m23.21500838s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.989912858 (13h52m23.215095904s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990129755 (13h52m23.215312812s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990276592 (13h52m23.215459649s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990389334 (13h52m23.215572381s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990591925 (13h52m23.215774982s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990601453 (13h52m23.21578451s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.990892651 (13h52m23.216075708s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991232150 (13h52m23.216415207s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991538705 (13h52m23.216721762s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.991834462 (13h52m23.217017519s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992188699 (13h52m23.217371756s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992502169 (13h52m23.217685215s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.992810137 (13h52m23.217993194s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993125952 (13h52m23.218309009s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993435885 (13h52m23.218618942s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:07.993730108 (13h52m23.218913165s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Uploading piece... Aug 26 05:56:08.074651266 (13h52m23.299834323s) [upload=12168]: Next returned nil segment Aug 26 05:56:08.074675180 (13h52m23.299858227s) [upload=12168]: Waiting for error group managing segments... Aug 26 05:56:08.082966317 (13h52m23.308149374s) [noise=true , address=45.91.116.14:4222 , piece_id=6PPXN4TJ4YYXU3K22Z3AI5MLBYU5GUNK5R2WDDMH4B3T46GJISDQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.082996353 (13h52m23.3081794s) [noise=true , address=45.91.116.8:4049 , piece_id=2EUJ3TSPUDEMLSF76ARFMPK45JNRO2NUZBO66YNWOLVVLISSYDNA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.083391117 (13h52m23.308574174s) [noise=true , address=45.91.116.6:4995 , piece_id=H6KYB7XX2DRLBDT23Y3K4VDCGUYG4FYAQ5I7VKSZ7W2FRVLNFRUA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084118455 (13h52m23.309301512s) [noise=true , address=45.91.116.7:4043 , piece_id=SPDDQ3TZ3JH6HLRO3X2PS2FNAYCPYYN6ESKLU5EICNVJKS3M6Q6A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.084749531 (13h52m23.309932578s) [noise=true , address=45.91.116.5:4310 , piece_id=MA63EJFR6MUPCX5U6F3RXTFCSAHDATLMUXO3HA6JXUV2HIWL2SFQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085037694 (13h52m23.310220751s) [noise=true , address=77.90.31.12:4624 , piece_id=LTR3SAVXHPF6KEOFAABU4QE7672XEGJEAUV2F7ACVH476B4I7XNQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.085112013 (13h52m23.31029507s) [noise=true , address=77.90.31.18:4830 , piece_id=QPZEQYF334LEU6AOVDCHJWJMCYLRWWUJRCQVQ4LHY6F2J62XB3IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.086754614 (13h52m23.311937671s) [noise=true , address=104.36.82.53:4149 , piece_id=RAZCHICI4DKV3T3Q7RRJWGV7X56TSIFCB3KCCEYZ7HLD54XY367A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087019111 (13h52m23.312202158s) [noise=true , address=77.90.31.14:4682 , piece_id=OWD7BW3LZOCO5PBFZHN37YEG3CEJUNBFN3Y2WQLUM4BRQZ6ENSJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087382184 (13h52m23.312565231s) [noise=true , address=104.36.82.60:4344 , piece_id=6S4MXAPHQKSAROQWVQKB7OPO65QQ5HE3SL7UJDYDYGXBC3VJQPVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.087977184 (13h52m23.313160241s) [noise=true , address=104.36.82.4:4835 , piece_id=ERFLECEXUOPAEDWZS7DNGIM4JQCRAXALCTMSX4CVCSXES6CRV3RA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088323485 (13h52m23.313506533s) [noise=true , address=104.36.82.23:4457 , piece_id=QCOSM6XWYJMG45K3KYGEXUAIH6G7K6QRDMVOWH4VS4VNGK4YWFVA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.088980170 (13h52m23.314163217s) [noise=true , address=104.36.82.9:4035 , piece_id=NKFUYDRAXQSREXB2YOCHSJXB24XBX2V2D3C7T52YZUVX4UVP33IQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089398397 (13h52m23.314581444s) [noise=true , address=104.36.82.20:4446 , piece_id=AESARGYI7K5FF44J2HYFGLMKDVYQEG6UKQYJG2XQSGCXDHMLUPEA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: <nil> Aug 26 05:56:08.089406202 (13h52m23.314589259s) [seg_pos={0 0}, upload=12168]: Segment reached optimal threshold of 14 pieces. Aug 26 05:56:08.089605186 (13h52m23.314788273s) [noise=true , address=104.36.82.5:4868 , piece_id=TNBNMCYYV7YSXTWSGQTSLN7CHWPVPO5ATZBEOEJMU7GPQFINUSRA, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:18Srfics6dFkQeHZ2WF5YkFuLm2719Pw4Xgg9ntk12KnTBcn8v): protocol: expected piece hash; context canceled; EOF Aug 26 05:56:08.089911312 (13h52m23.315094369s) [noise=true , address=79.127.205.234:4616 , piece_id=RYQ7HGG4FRP6J2MFRJPX3GZTA66ZMFUY4Y3H67NFZCTKWLD3OKJQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:52878->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089946328 (13h52m23.315129385s) [noise=true , address=79.127.205.245:4247 , piece_id=DMCHX73ODUFLH3RW5PZOTCVJIC7TWM26QU5I4GNIQCB3FRKP7RLQ, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Xo8UnshyP8jAYUTtaCKnVZKbUXFvCTJNYsshmMk97dH4FvM33): protocol: read tcp 5.161.89.252:64714->79.127.205.245:4247: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089977206 (13h52m23.315160263s) [noise=true , address=79.127.201.215:4343 , piece_id=AGMITCTK334DTMR4KUCDNS2L5HHWLNRKCPM76ZUV5GJWWP3GYR2A, seg_pos={0 0}, upload=12168]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1NNN3TCTNoLCefjYxwroGgXt4XXxDWo4yGCtbrm7FfEZsMGriU): protocol: read tcp 5.161.89.252:52396->79.127.201.215:4343: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 26 05:56:08.089984850 (13h52m23.315167898s) [seg_pos={0 0}, upload=12168]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 26 05:56:08.090031578 (13h52m23.315214625s) [upload=12168]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 26 05:56:08.090034894 (13h52m23.315217942s) [upload=12168]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 05:56:08.350683461 (13h52m23.575866518s) [upload=12168]: Done uploading segments... Aug 26 05:56:08.350709680 (13h52m23.575892737s) [upload=12168]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration: 1.258133011s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...XX.X........XX.XX.XX.....................................] duration: 1.703949465s FAIL (2025-08-26 06:26:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.703972868s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline) antiblip: 3 out of 60 is failed (threshold=3) [..XX...................X....................................] duration: 164.389612ms FAIL (2025-08-26 06:26:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:26.631953597 (13h56m41.857136654s) [upload=12237]: Starting upload Aug 26 06:00:26.632024379 (13h56m41.857207426s) [upload=12237]: Done starting upload Aug 26 06:00:26.632076819 (13h56m41.857259876s) [upload=12237]: Uploading segments... Aug 26 06:00:26.632123506 (13h56m41.857306564s) [upload=12237]: Got next segment. Inline: true Aug 26 06:00:26.632128556 (13h56m41.857311613s) [upload=12237]: Waiting for error group managing segments... Aug 26 06:00:26.632165375 (13h56m41.857348432s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:26.632169212 (13h56m41.857352269s) [upload=12237]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:26.632171647 (13h56m41.857354704s) [upload=12237]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:26.918443627 (13h56m42.143626694s) [upload=12237]: Done uploading segments... Aug 26 06:00:26.918470347 (13h56m42.143653424s) [upload=12237]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 164.409359ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 8 out of 60 is failed (threshold=3) [XX.............X.X.X.XXX....................................] duration: 145.139217ms FAIL (2025-08-26 06:26:42 UTC) orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded Duration: 145.166959ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 6 out of 60 is failed (threshold=3) [XX..................XX.X.X..................................] duration: 148.161221ms FAIL (2025-08-26 06:26:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled ======LOGS====== Aug 26 06:00:12.457863229 (13h56m27.683046286s) [upload=12233]: Starting upload Aug 26 06:00:12.457912863 (13h56m27.68309592s) [upload=12233]: Done starting upload Aug 26 06:00:12.457952868 (13h56m27.683135925s) [upload=12233]: Uploading segments... Aug 26 06:00:12.457977694 (13h56m27.683160752s) [upload=12233]: Got next segment. Inline: true Aug 26 06:00:12.457981241 (13h56m27.683164298s) [upload=12233]: Waiting for error group managing segments... Aug 26 06:00:12.458011648 (13h56m27.683194705s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 26 06:00:12.458015195 (13h56m27.683198241s) [upload=12233]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 26 06:00:12.458017880 (13h56m27.683200937s) [upload=12233]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 26 06:00:22.115296668 (13h56m37.340479716s) [upload=12233]: Done uploading segments... Aug 26 06:00:22.115310454 (13h56m37.340493511s) [upload=12233]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled Duration:* 148.188252ms core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [....XX.XX.........X.XX.X.X..X...............................] duration: 343.915328ms FAIL (2025-08-26 06:26:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 343.93727ms edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 6 out of 60 is failed (threshold=3) [...XXX..............XX.X....................................] duration: 168.235258ms FAIL (2025-08-26 06:26:30 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 168.311931ms edge gateway us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing) antiblip: 1 out of 60 is failed (threshold=6) [.........................X..................................] duration: 38.552115ms remote: "tcp:136.0.77.2:443" response headers: Access-Control-Allow-Methods=GET, HEAD Access-Control-Allow-Origin=* Date=Tue, 26 Aug 2025 05:58:10 GMT Accept-Ranges=bytes X-Request-Id=2dEEZwg2VX7 Content-Type=text/html; charset=utf-8 Access-Control-Allow-Headers=* target: https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt FAIL (2025-08-26 06:26:25 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n <meta charset=\"utf-8\">\n <title>Error \| Storj</title>\n\n <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n <meta property=\"og:title\" content=\"Error \| Storj\" />\n <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n \n <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <meta name=\"twitter:card\" content=\"summary\" />\n <meta name=\"twitter:site\" content=\"@storj\" />\n <meta name=\"twitter:title\" content=\"Error \| Storj\" />\n <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n \n <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n \n\n <link rel=\"shortcut icon\"\n href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY" Duration: 38.570761ms edge linksharing bgp us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)

checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://uptime-checks-us-select.s3.us-select-1.storjapi.io/hello-world.txt?AWSAccessKeyId=jv6fvdi5muos6nw22tdksxgfrd2a&Signature=KjTmfuFyjkIAPapJ1pe5FKXvyIY%3D&Expires=1771562410", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us_select_1.gateway)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)

checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)