Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 118930 reports (Page 1333 of 2379)
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [...............................................X....XX......]
duration:
2.978511206s
FAIL (2025-08-26 05:05:03 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F386242827DE1, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 2.978528317s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [............................................XX.......X......]
duration:
638.304588ms
FAIL (2025-08-26 05:04:54 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:59:12.156304404 (12h55m27.381487481s) [upload=11354]: Starting upload
Aug 26 04:59:12.156450137 (12h55m27.381633214s) [upload=11354]: Done starting upload
Aug 26 04:59:12.156550065 (12h55m27.381733152s) [upload=11354]: Uploading segments...
Aug 26 04:59:12.156613884 (12h55m27.381796961s) [upload=11354]: Got next segment. Inline: true
Aug 26 04:59:12.156623683 (12h55m27.381806759s) [upload=11354]: Waiting for error group managing segments...
Aug 26 04:59:12.156689306 (12h55m27.381872394s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:59:12.156697092 (12h55m27.381880168s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 04:59:12.156703453 (12h55m27.381886561s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:42.137624713 (12h55m57.36280777s) [upload=11354]: Done uploading segments...
Aug 26 04:59:42.137641224 (12h55m57.362824281s) [upload=11354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 638.345124ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
4 out of 60 is failed (threshold=3) [............................................XX.......X..X...]
duration:
3.043531585s
FAIL (2025-08-26 05:04:45 UTC)
operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 3.043551162s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [.............................................X.....XX.......]
duration:
886.494756ms
FAIL (2025-08-26 05:04:35 UTC)
operation: Commit
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 886.521086ms core uplink us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
209.983685ms
remote:
"tcp:136.0.77.2:443"
response headers:
Server=Storj
X-Amz-Request-Id=185F3861EA4B077B
X-Request-Id=5H4AYXsA7rp
Content-Security-Policy=block-all-mixed-content
Vary=Origin
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 04:59:33 GMT
Accept-Ranges=bytes
Content-Length=423
Content-Type=application/xml
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 05:04:30 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3861EA4B077B</RequestId><HostId></HostId></Error>"
Duration: 210.01813ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
1.641173407s
FAIL (2025-08-26 05:04:29 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F381EFB1C9300, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 1.641197803s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..............................................X.....X.......]
duration:
2.124733698s
FAIL (2025-08-26 05:04:26 UTC)
orbiter warning: 
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:58:54.632991348 (12h55m9.858174394s) [upload=11349]: Starting upload
Aug 26 04:58:54.633055548 (12h55m9.858238594s) [upload=11349]: Done starting upload
Aug 26 04:58:54.633126050 (12h55m9.858309107s) [upload=11349]: Uploading segments...
Aug 26 04:58:54.633283045 (12h55m9.858466102s) [upload=11349]: Got next segment. Inline: false
Aug 26 04:58:54.633316488 (12h55m9.858499545s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:58:54.633321207 (12h55m9.858504264s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:58:58.655965649 (12h55m13.881148726s) [seg_pos={0 0}, upload=11349]: Begin upload segment...
Aug 26 04:58:58.656054146 (12h55m13.881237223s) [seg_pos={0 0}, upload=11349]: Done begin upload segment.
Aug 26 04:58:58.656098769 (12h55m13.881281847s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656148543 (12h55m13.881331611s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656574924 (12h55m13.881757991s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656741646 (12h55m13.881924724s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656978031 (12h55m13.882161108s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657326154 (12h55m13.882509232s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657343688 (12h55m13.882526764s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657744651 (12h55m13.882927788s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657800345 (12h55m13.882983422s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658144241 (12h55m13.883327318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658262203 (12h55m13.883445279s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658527862 (12h55m13.883710939s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658722727 (12h55m13.883905804s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658883790 (12h55m13.884066867s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659220493 (12h55m13.884403569s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659753644 (12h55m13.884936711s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.754621259 (12h55m13.979804306s) [upload=11349]: Next returned nil segment
Aug 26 04:58:58.754644293 (12h55m13.97982734s) [upload=11349]: Waiting for error group managing segments...
Aug 26 04:58:58.763291261 (12h55m13.988474318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764054023 (12h55m13.98923707s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764064784 (12h55m13.989247841s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765165831 (12h55m13.990348879s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765174608 (12h55m13.990357665s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765575150 (12h55m13.990758197s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765615426 (12h55m13.990798483s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765940345 (12h55m13.991123402s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766080359 (12h55m13.991263406s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766966263 (12h55m13.992149309s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.768385056 (12h55m13.993568093s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.769690068 (12h55m13.994873115s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772653423 (12h55m13.99783646s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772672168 (12h55m13.997855215s) [seg_pos={0 0}, upload=11349]: Segment reached optimal threshold of 13 pieces.
Aug 26 04:58:58.772774941 (12h55m13.997957988s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772793706 (12h55m13.997976753s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772839091 (12h55m13.998022138s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772880569 (12h55m13.998063606s) [seg_pos={0 0}, upload=11349]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 04:58:58.772940883 (12h55m13.99812392s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:58:58.772944299 (12h55m13.998127347s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:06.816180513 (12h55m22.041363581s) [upload=11349]: Done uploading segments...
Aug 26 04:59:06.816202064 (12h55m22.041385141s) [upload=11349]: Upload finished. err: <nil>
Duration: 2.124762292s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
2.161217803s
FAIL (2025-08-26 05:04:23 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
n
Aug 26 04:59:54.491876868 (12h56m9.717059954s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:59:55.898473672 (12h56m11.123656729s) [seg_pos={0 0}, upload=11365]: Begin upload segment...
Aug 26 04:59:55.898578450 (12h56m11.123761507s) [seg_pos={0 0}, upload=11365]: Done begin upload segment.
Aug 26 04:59:55.898604639 (12h56m11.123787696s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898647179 (12h56m11.123830226s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898996365 (12h56m11.124179412s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899235543 (12h56m11.1244186s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899248037 (12h56m11.124431094s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899506833 (12h56m11.12468988s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899574300 (12h56m11.124757347s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899843586 (12h56m11.125026643s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899963090 (12h56m11.125146147s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900138228 (12h56m11.125321285s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900431890 (12h56m11.125614947s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900558217 (12h56m11.125741264s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900861106 (12h56m11.126044163s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900871085 (12h56m11.126054142s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901238495 (12h56m11.126421542s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901672420 (12h56m11.126855466s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.902022848 (12h56m11.127205905s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.903391288 (12h56m11.128574334s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.994352287 (12h56m11.219535334s) [upload=11365]: Next returned nil segment
Aug 26 04:59:55.994374639 (12h56m11.219557687s) [upload=11365]: Waiting for error group managing segments...
Aug 26 04:59:56.001240734 (12h56m11.226423791s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.002465303 (12h56m11.227648349s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003141202 (12h56m11.228324259s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003546413 (12h56m11.22872948s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003630461 (12h56m11.228813518s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004292123 (12h56m11.22947517s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004703165 (12h56m11.229886222s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.006769947 (12h56m11.231952993s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.007034093 (12h56m11.23221715s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.008777136 (12h56m11.233960183s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.011187783 (12h56m11.236370821s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.017859823 (12h56m11.24304287s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027329847 (12h56m11.252512894s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027997051 (12h56m11.253180087s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.028028038 (12h56m11.253211096s) [seg_pos={0 0}, upload=11365]: Segment reached optimal threshold of 14 pieces.
Aug 26 04:59:56.028506516 (12h56m11.253689574s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ofwUBJwLG7eUYo7JE1ufwxRRyQ9HUpTZnshuTz6YF2JMfzVcX): protocol: rpc: dial tcp 79.127.205.245:4249: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028671135 (12h56m11.253854192s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ea2CWMvyMotcFqC6Q1vf2R5UWWZFgwPcmSdkSqMAwCQiVHdoM): protocol: read tcp 5.161.89.252:33180->79.127.205.234:4614: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028792494 (12h56m11.253975541s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XmGcLqbv3ctCx52m42uKqwKpbWjgzEbvoJmdRLXt81qiGvY6M): protocol: read tcp 5.161.89.252:41050->64.49.13.170:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028849591 (12h56m11.254032638s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: read tcp 5.161.89.252:26264->79.127.201.215:4349: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028882292 (12h56m11.25406534s) [seg_pos={0 0}, upload=11365]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 26 04:59:56.028946543 (12h56m11.25412959s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:59:56.028951021 (12h56m11.254134069s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:56.240070251 (12h56m11.465253308s) [upload=11365]: Done uploading segments...
Aug 26 04:59:56.240108883 (12h56m11.46529194s) [upload=11365]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 2.16123768s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
1 out of 60 is failed (threshold=6) [....................................................X.......]
duration:
233.334458ms
remote:
"tcp:136.0.77.2:443"
response headers:
Date=Tue, 26 Aug 2025 04:59:41 GMT
Access-Control-Allow-Headers=*
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Content-Type=text/html; charset=utf-8
Accept-Ranges=bytes
X-Request-Id=YDWvXrmwXNN
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 05:04:14 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 233.347402ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
4 out of 60 is failed (threshold=3) [.............................................XX.......X..X..]
duration:
4.507545296s
FAIL (2025-08-26 05:04:10 UTC)
operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 4.507565605s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...............................................X.....X......]
duration:
974.160619ms
FAIL (2025-08-26 05:04:07 UTC)
orbiter warning: 
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:58:54.632991348 (12h55m9.858174394s) [upload=11349]: Starting upload
Aug 26 04:58:54.633055548 (12h55m9.858238594s) [upload=11349]: Done starting upload
Aug 26 04:58:54.633126050 (12h55m9.858309107s) [upload=11349]: Uploading segments...
Aug 26 04:58:54.633283045 (12h55m9.858466102s) [upload=11349]: Got next segment. Inline: false
Aug 26 04:58:54.633316488 (12h55m9.858499545s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:58:54.633321207 (12h55m9.858504264s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:58:58.655965649 (12h55m13.881148726s) [seg_pos={0 0}, upload=11349]: Begin upload segment...
Aug 26 04:58:58.656054146 (12h55m13.881237223s) [seg_pos={0 0}, upload=11349]: Done begin upload segment.
Aug 26 04:58:58.656098769 (12h55m13.881281847s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656148543 (12h55m13.881331611s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656574924 (12h55m13.881757991s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656741646 (12h55m13.881924724s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656978031 (12h55m13.882161108s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657326154 (12h55m13.882509232s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657343688 (12h55m13.882526764s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657744651 (12h55m13.882927788s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657800345 (12h55m13.882983422s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658144241 (12h55m13.883327318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658262203 (12h55m13.883445279s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658527862 (12h55m13.883710939s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658722727 (12h55m13.883905804s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658883790 (12h55m13.884066867s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659220493 (12h55m13.884403569s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659753644 (12h55m13.884936711s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.754621259 (12h55m13.979804306s) [upload=11349]: Next returned nil segment
Aug 26 04:58:58.754644293 (12h55m13.97982734s) [upload=11349]: Waiting for error group managing segments...
Aug 26 04:58:58.763291261 (12h55m13.988474318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764054023 (12h55m13.98923707s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764064784 (12h55m13.989247841s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765165831 (12h55m13.990348879s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765174608 (12h55m13.990357665s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765575150 (12h55m13.990758197s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765615426 (12h55m13.990798483s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765940345 (12h55m13.991123402s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766080359 (12h55m13.991263406s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766966263 (12h55m13.992149309s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.768385056 (12h55m13.993568093s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.769690068 (12h55m13.994873115s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772653423 (12h55m13.99783646s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772672168 (12h55m13.997855215s) [seg_pos={0 0}, upload=11349]: Segment reached optimal threshold of 13 pieces.
Aug 26 04:58:58.772774941 (12h55m13.997957988s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772793706 (12h55m13.997976753s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772839091 (12h55m13.998022138s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772880569 (12h55m13.998063606s) [seg_pos={0 0}, upload=11349]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 04:58:58.772940883 (12h55m13.99812392s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:58:58.772944299 (12h55m13.998127347s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:06.816180513 (12h55m22.041363581s) [upload=11349]: Done uploading segments...
Aug 26 04:59:06.816202064 (12h55m22.041385141s) [upload=11349]: Upload finished. err: <nil>
Duration: 974.203049ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [..............................................X.....XX......]
duration:
893.619366ms
FAIL (2025-08-26 05:04:03 UTC)
operation: Commit
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 893.645165ms core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [................................................X....XX.....]
duration:
2.978919281s
FAIL (2025-08-26 05:04:02 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F386242827DE1, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 2.978981569s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
725.382777ms
FAIL (2025-08-26 05:03:53 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F381EFB1C9300, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 725.403175ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [.............................................XX.......X.....]
duration:
826.038745ms
FAIL (2025-08-26 05:03:46 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:59:12.156304404 (12h55m27.381487481s) [upload=11354]: Starting upload
Aug 26 04:59:12.156450137 (12h55m27.381633214s) [upload=11354]: Done starting upload
Aug 26 04:59:12.156550065 (12h55m27.381733152s) [upload=11354]: Uploading segments...
Aug 26 04:59:12.156613884 (12h55m27.381796961s) [upload=11354]: Got next segment. Inline: true
Aug 26 04:59:12.156623683 (12h55m27.381806759s) [upload=11354]: Waiting for error group managing segments...
Aug 26 04:59:12.156689306 (12h55m27.381872394s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:59:12.156697092 (12h55m27.381880168s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 04:59:12.156703453 (12h55m27.381886561s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:42.137624713 (12h55m57.36280777s) [upload=11354]: Done uploading segments...
Aug 26 04:59:42.137641224 (12h55m57.362824281s) [upload=11354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 826.082677ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
1.561405125s
FAIL (2025-08-26 05:03:36 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
n
Aug 26 04:59:54.491876868 (12h56m9.717059954s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:59:55.898473672 (12h56m11.123656729s) [seg_pos={0 0}, upload=11365]: Begin upload segment...
Aug 26 04:59:55.898578450 (12h56m11.123761507s) [seg_pos={0 0}, upload=11365]: Done begin upload segment.
Aug 26 04:59:55.898604639 (12h56m11.123787696s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898647179 (12h56m11.123830226s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898996365 (12h56m11.124179412s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899235543 (12h56m11.1244186s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899248037 (12h56m11.124431094s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899506833 (12h56m11.12468988s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899574300 (12h56m11.124757347s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899843586 (12h56m11.125026643s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899963090 (12h56m11.125146147s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900138228 (12h56m11.125321285s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900431890 (12h56m11.125614947s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900558217 (12h56m11.125741264s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900861106 (12h56m11.126044163s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900871085 (12h56m11.126054142s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901238495 (12h56m11.126421542s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901672420 (12h56m11.126855466s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.902022848 (12h56m11.127205905s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.903391288 (12h56m11.128574334s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.994352287 (12h56m11.219535334s) [upload=11365]: Next returned nil segment
Aug 26 04:59:55.994374639 (12h56m11.219557687s) [upload=11365]: Waiting for error group managing segments...
Aug 26 04:59:56.001240734 (12h56m11.226423791s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.002465303 (12h56m11.227648349s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003141202 (12h56m11.228324259s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003546413 (12h56m11.22872948s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003630461 (12h56m11.228813518s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004292123 (12h56m11.22947517s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004703165 (12h56m11.229886222s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.006769947 (12h56m11.231952993s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.007034093 (12h56m11.23221715s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.008777136 (12h56m11.233960183s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.011187783 (12h56m11.236370821s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.017859823 (12h56m11.24304287s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027329847 (12h56m11.252512894s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027997051 (12h56m11.253180087s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.028028038 (12h56m11.253211096s) [seg_pos={0 0}, upload=11365]: Segment reached optimal threshold of 14 pieces.
Aug 26 04:59:56.028506516 (12h56m11.253689574s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ofwUBJwLG7eUYo7JE1ufwxRRyQ9HUpTZnshuTz6YF2JMfzVcX): protocol: rpc: dial tcp 79.127.205.245:4249: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028671135 (12h56m11.253854192s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ea2CWMvyMotcFqC6Q1vf2R5UWWZFgwPcmSdkSqMAwCQiVHdoM): protocol: read tcp 5.161.89.252:33180->79.127.205.234:4614: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028792494 (12h56m11.253975541s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XmGcLqbv3ctCx52m42uKqwKpbWjgzEbvoJmdRLXt81qiGvY6M): protocol: read tcp 5.161.89.252:41050->64.49.13.170:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028849591 (12h56m11.254032638s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: read tcp 5.161.89.252:26264->79.127.201.215:4349: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028882292 (12h56m11.25406534s) [seg_pos={0 0}, upload=11365]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 26 04:59:56.028946543 (12h56m11.25412959s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:59:56.028951021 (12h56m11.254134069s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:56.240070251 (12h56m11.465253308s) [upload=11365]: Done uploading segments...
Aug 26 04:59:56.240108883 (12h56m11.46529194s) [upload=11365]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 1.561432937s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
1 out of 60 is failed (threshold=6) [.....................................................X......]
duration:
235.383366ms
remote:
"tcp:136.0.77.2:443"
response headers:
Date=Tue, 26 Aug 2025 04:59:41 GMT
Access-Control-Allow-Headers=*
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Content-Type=text/html; charset=utf-8
Accept-Ranges=bytes
X-Request-Id=YDWvXrmwXNN
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 05:03:34 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 235.405619ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [................................................X.....X.....]
duration:
3.312424713s
FAIL (2025-08-26 05:03:30 UTC)
orbiter warning: 
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:58:54.632991348 (12h55m9.858174394s) [upload=11349]: Starting upload
Aug 26 04:58:54.633055548 (12h55m9.858238594s) [upload=11349]: Done starting upload
Aug 26 04:58:54.633126050 (12h55m9.858309107s) [upload=11349]: Uploading segments...
Aug 26 04:58:54.633283045 (12h55m9.858466102s) [upload=11349]: Got next segment. Inline: false
Aug 26 04:58:54.633316488 (12h55m9.858499545s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:58:54.633321207 (12h55m9.858504264s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:58:58.655965649 (12h55m13.881148726s) [seg_pos={0 0}, upload=11349]: Begin upload segment...
Aug 26 04:58:58.656054146 (12h55m13.881237223s) [seg_pos={0 0}, upload=11349]: Done begin upload segment.
Aug 26 04:58:58.656098769 (12h55m13.881281847s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656148543 (12h55m13.881331611s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656574924 (12h55m13.881757991s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656741646 (12h55m13.881924724s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656978031 (12h55m13.882161108s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657326154 (12h55m13.882509232s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657343688 (12h55m13.882526764s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657744651 (12h55m13.882927788s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657800345 (12h55m13.882983422s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658144241 (12h55m13.883327318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658262203 (12h55m13.883445279s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658527862 (12h55m13.883710939s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658722727 (12h55m13.883905804s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658883790 (12h55m13.884066867s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659220493 (12h55m13.884403569s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659753644 (12h55m13.884936711s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.754621259 (12h55m13.979804306s) [upload=11349]: Next returned nil segment
Aug 26 04:58:58.754644293 (12h55m13.97982734s) [upload=11349]: Waiting for error group managing segments...
Aug 26 04:58:58.763291261 (12h55m13.988474318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764054023 (12h55m13.98923707s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764064784 (12h55m13.989247841s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765165831 (12h55m13.990348879s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765174608 (12h55m13.990357665s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765575150 (12h55m13.990758197s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765615426 (12h55m13.990798483s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765940345 (12h55m13.991123402s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766080359 (12h55m13.991263406s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766966263 (12h55m13.992149309s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.768385056 (12h55m13.993568093s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.769690068 (12h55m13.994873115s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772653423 (12h55m13.99783646s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772672168 (12h55m13.997855215s) [seg_pos={0 0}, upload=11349]: Segment reached optimal threshold of 13 pieces.
Aug 26 04:58:58.772774941 (12h55m13.997957988s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772793706 (12h55m13.997976753s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772839091 (12h55m13.998022138s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772880569 (12h55m13.998063606s) [seg_pos={0 0}, upload=11349]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 04:58:58.772940883 (12h55m13.99812392s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:58:58.772944299 (12h55m13.998127347s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:06.816180513 (12h55m22.041363581s) [upload=11349]: Done uploading segments...
Aug 26 04:59:06.816202064 (12h55m22.041385141s) [upload=11349]: Upload finished. err: <nil>
Duration: 3.312445703s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
3.136957545s
FAIL (2025-08-26 05:03:28 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F381EFB1C9300, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 3.1369762s edge gateway us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
204.260661ms
remote:
"tcp:136.0.77.2:443"
response headers:
Server=Storj
X-Amz-Request-Id=185F3861EA4B077B
X-Request-Id=5H4AYXsA7rp
Content-Security-Policy=block-all-mixed-content
Vary=Origin
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 04:59:33 GMT
Accept-Ranges=bytes
Content-Length=423
Content-Type=application/xml
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 05:03:21 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3861EA4B077B</RequestId><HostId></HostId></Error>"
Duration: 204.301806ms edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [.................................................X....XX....]
duration:
3.16511058s
FAIL (2025-08-26 05:03:16 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F386242827DE1, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 3.165127802s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [...............................................X.....XX.....]
duration:
1.204530327s
FAIL (2025-08-26 05:03:13 UTC)
operation: Commit
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 1.204609746s core uplink us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
1 out of 60 is failed (threshold=6) [......................................................X.....]
duration:
221.291954ms
remote:
"tcp:136.0.77.2:443"
response headers:
Date=Tue, 26 Aug 2025 04:59:41 GMT
Access-Control-Allow-Headers=*
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Content-Type=text/html; charset=utf-8
Accept-Ranges=bytes
X-Request-Id=YDWvXrmwXNN
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 05:03:09 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 221.318536ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
4 out of 60 is failed (threshold=3) [..............................................XX.......X..X.]
duration:
3.231467015s
FAIL (2025-08-26 05:03:06 UTC)
operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 3.231497963s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
20.99845167s
FAIL (2025-08-26 05:02:57 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
n
Aug 26 04:59:54.491876868 (12h56m9.717059954s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:59:55.898473672 (12h56m11.123656729s) [seg_pos={0 0}, upload=11365]: Begin upload segment...
Aug 26 04:59:55.898578450 (12h56m11.123761507s) [seg_pos={0 0}, upload=11365]: Done begin upload segment.
Aug 26 04:59:55.898604639 (12h56m11.123787696s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898647179 (12h56m11.123830226s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898996365 (12h56m11.124179412s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899235543 (12h56m11.1244186s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899248037 (12h56m11.124431094s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899506833 (12h56m11.12468988s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899574300 (12h56m11.124757347s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899843586 (12h56m11.125026643s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899963090 (12h56m11.125146147s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900138228 (12h56m11.125321285s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900431890 (12h56m11.125614947s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900558217 (12h56m11.125741264s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900861106 (12h56m11.126044163s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900871085 (12h56m11.126054142s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901238495 (12h56m11.126421542s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901672420 (12h56m11.126855466s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.902022848 (12h56m11.127205905s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.903391288 (12h56m11.128574334s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.994352287 (12h56m11.219535334s) [upload=11365]: Next returned nil segment
Aug 26 04:59:55.994374639 (12h56m11.219557687s) [upload=11365]: Waiting for error group managing segments...
Aug 26 04:59:56.001240734 (12h56m11.226423791s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.002465303 (12h56m11.227648349s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003141202 (12h56m11.228324259s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003546413 (12h56m11.22872948s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003630461 (12h56m11.228813518s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004292123 (12h56m11.22947517s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004703165 (12h56m11.229886222s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.006769947 (12h56m11.231952993s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.007034093 (12h56m11.23221715s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.008777136 (12h56m11.233960183s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.011187783 (12h56m11.236370821s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.017859823 (12h56m11.24304287s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027329847 (12h56m11.252512894s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027997051 (12h56m11.253180087s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.028028038 (12h56m11.253211096s) [seg_pos={0 0}, upload=11365]: Segment reached optimal threshold of 14 pieces.
Aug 26 04:59:56.028506516 (12h56m11.253689574s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ofwUBJwLG7eUYo7JE1ufwxRRyQ9HUpTZnshuTz6YF2JMfzVcX): protocol: rpc: dial tcp 79.127.205.245:4249: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028671135 (12h56m11.253854192s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ea2CWMvyMotcFqC6Q1vf2R5UWWZFgwPcmSdkSqMAwCQiVHdoM): protocol: read tcp 5.161.89.252:33180->79.127.205.234:4614: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028792494 (12h56m11.253975541s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XmGcLqbv3ctCx52m42uKqwKpbWjgzEbvoJmdRLXt81qiGvY6M): protocol: read tcp 5.161.89.252:41050->64.49.13.170:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028849591 (12h56m11.254032638s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: read tcp 5.161.89.252:26264->79.127.201.215:4349: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028882292 (12h56m11.25406534s) [seg_pos={0 0}, upload=11365]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 26 04:59:56.028946543 (12h56m11.25412959s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:59:56.028951021 (12h56m11.254134069s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:56.240070251 (12h56m11.465253308s) [upload=11365]: Done uploading segments...
Aug 26 04:59:56.240108883 (12h56m11.46529194s) [upload=11365]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 20.998473211s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [................................................X.....XX....]
duration:
29.010669825s
FAIL (2025-08-26 05:02:55 UTC)
operation: Commit
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 29.010697116s core uplink us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
5.427016396s
remote:
"tcp:136.0.77.2:443"
response headers:
Server=Storj
X-Amz-Request-Id=185F3861EA4B077B
X-Request-Id=5H4AYXsA7rp
Content-Security-Policy=block-all-mixed-content
Vary=Origin
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 04:59:33 GMT
Accept-Ranges=bytes
Content-Length=423
Content-Type=application/xml
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 05:02:55 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3861EA4B077B</RequestId><HostId></HostId></Error>"
Duration: 5.427035993s edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.................................................X.....X....]
duration:
29.550956047s
FAIL (2025-08-26 05:02:50 UTC)
orbiter warning: 
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:58:54.632991348 (12h55m9.858174394s) [upload=11349]: Starting upload
Aug 26 04:58:54.633055548 (12h55m9.858238594s) [upload=11349]: Done starting upload
Aug 26 04:58:54.633126050 (12h55m9.858309107s) [upload=11349]: Uploading segments...
Aug 26 04:58:54.633283045 (12h55m9.858466102s) [upload=11349]: Got next segment. Inline: false
Aug 26 04:58:54.633316488 (12h55m9.858499545s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:58:54.633321207 (12h55m9.858504264s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:58:58.655965649 (12h55m13.881148726s) [seg_pos={0 0}, upload=11349]: Begin upload segment...
Aug 26 04:58:58.656054146 (12h55m13.881237223s) [seg_pos={0 0}, upload=11349]: Done begin upload segment.
Aug 26 04:58:58.656098769 (12h55m13.881281847s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656148543 (12h55m13.881331611s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656574924 (12h55m13.881757991s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656741646 (12h55m13.881924724s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656978031 (12h55m13.882161108s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657326154 (12h55m13.882509232s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657343688 (12h55m13.882526764s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657744651 (12h55m13.882927788s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657800345 (12h55m13.882983422s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658144241 (12h55m13.883327318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658262203 (12h55m13.883445279s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658527862 (12h55m13.883710939s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658722727 (12h55m13.883905804s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658883790 (12h55m13.884066867s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659220493 (12h55m13.884403569s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659753644 (12h55m13.884936711s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.754621259 (12h55m13.979804306s) [upload=11349]: Next returned nil segment
Aug 26 04:58:58.754644293 (12h55m13.97982734s) [upload=11349]: Waiting for error group managing segments...
Aug 26 04:58:58.763291261 (12h55m13.988474318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764054023 (12h55m13.98923707s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764064784 (12h55m13.989247841s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765165831 (12h55m13.990348879s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765174608 (12h55m13.990357665s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765575150 (12h55m13.990758197s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765615426 (12h55m13.990798483s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765940345 (12h55m13.991123402s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766080359 (12h55m13.991263406s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766966263 (12h55m13.992149309s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.768385056 (12h55m13.993568093s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.769690068 (12h55m13.994873115s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772653423 (12h55m13.99783646s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772672168 (12h55m13.997855215s) [seg_pos={0 0}, upload=11349]: Segment reached optimal threshold of 13 pieces.
Aug 26 04:58:58.772774941 (12h55m13.997957988s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772793706 (12h55m13.997976753s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772839091 (12h55m13.998022138s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772880569 (12h55m13.998063606s) [seg_pos={0 0}, upload=11349]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 04:58:58.772940883 (12h55m13.99812392s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:58:58.772944299 (12h55m13.998127347s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:06.816180513 (12h55m22.041363581s) [upload=11349]: Done uploading segments...
Aug 26 04:59:06.816202064 (12h55m22.041385141s) [upload=11349]: Upload finished. err: <nil>
Duration: 29.550988809s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [..............................................XX.......X....]
duration:
1.577800155s
FAIL (2025-08-26 05:02:46 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:59:12.156304404 (12h55m27.381487481s) [upload=11354]: Starting upload
Aug 26 04:59:12.156450137 (12h55m27.381633214s) [upload=11354]: Done starting upload
Aug 26 04:59:12.156550065 (12h55m27.381733152s) [upload=11354]: Uploading segments...
Aug 26 04:59:12.156613884 (12h55m27.381796961s) [upload=11354]: Got next segment. Inline: true
Aug 26 04:59:12.156623683 (12h55m27.381806759s) [upload=11354]: Waiting for error group managing segments...
Aug 26 04:59:12.156689306 (12h55m27.381872394s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:59:12.156697092 (12h55m27.381880168s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 04:59:12.156703453 (12h55m27.381886561s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:42.137624713 (12h55m57.36280777s) [upload=11354]: Done uploading segments...
Aug 26 04:59:42.137641224 (12h55m57.362824281s) [upload=11354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1.577852092s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [..................................................X....XX...]
duration:
10.173746379s
FAIL (2025-08-26 05:02:38 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F386242827DE1, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 10.173867377s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
4 out of 60 is failed (threshold=3) [...............................................XX.......X..X]
duration:
44.883023417s
FAIL (2025-08-26 05:02:35 UTC)
operation: Commit
uplink: metaclient: context deadline exceeded
Duration: 44.883094169s core uplink us1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
1 out of 60 is failed (threshold=6) [.......................................................X....]
duration:
1.325320407s
remote:
"tcp:136.0.77.2:443"
response headers:
Date=Tue, 26 Aug 2025 04:59:41 GMT
Access-Control-Allow-Headers=*
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Content-Type=text/html; charset=utf-8
Accept-Ranges=bytes
X-Request-Id=YDWvXrmwXNN
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 05:02:28 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 1.325338389s edge linksharing bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
6.20960315s
FAIL (2025-08-26 05:02:18 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F381EFB1C9300, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 6.20961987s edge gateway us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
1 out of 60 is failed (threshold=6) [........................................................X...]
duration:
2.778466784s
remote:
"tcp:136.0.77.2:443"
response headers:
Date=Tue, 26 Aug 2025 04:59:41 GMT
Access-Control-Allow-Headers=*
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Content-Type=text/html; charset=utf-8
Accept-Ranges=bytes
X-Request-Id=YDWvXrmwXNN
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 05:02:11 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 2.778492242s edge linksharing bgp us1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
388.078858ms
remote:
"tcp:136.0.77.2:443"
response headers:
Server=Storj
X-Amz-Request-Id=185F3861EA4B077B
X-Request-Id=5H4AYXsA7rp
Content-Security-Policy=block-all-mixed-content
Vary=Origin
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 04:59:33 GMT
Accept-Ranges=bytes
Content-Length=423
Content-Type=application/xml
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 05:02:03 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3861EA4B077B</RequestId><HostId></HostId></Error>"
Duration: 388.099587ms edge gateway vhost bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [...............................................XX.......X...]
duration:
20.194817717s
FAIL (2025-08-26 05:02:02 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:59:12.156304404 (12h55m27.381487481s) [upload=11354]: Starting upload
Aug 26 04:59:12.156450137 (12h55m27.381633214s) [upload=11354]: Done starting upload
Aug 26 04:59:12.156550065 (12h55m27.381733152s) [upload=11354]: Uploading segments...
Aug 26 04:59:12.156613884 (12h55m27.381796961s) [upload=11354]: Got next segment. Inline: true
Aug 26 04:59:12.156623683 (12h55m27.381806759s) [upload=11354]: Waiting for error group managing segments...
Aug 26 04:59:12.156689306 (12h55m27.381872394s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:59:12.156697092 (12h55m27.381880168s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 04:59:12.156703453 (12h55m27.381886561s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:42.137624713 (12h55m57.36280777s) [upload=11354]: Done uploading segments...
Aug 26 04:59:42.137641224 (12h55m57.362824281s) [upload=11354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 20.194840159s core uplink us-select-1
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
8.117550604s
remote:
"tcp:136.0.77.2:443"
response headers:
Server=Storj
X-Amz-Request-Id=185F3861EA4B077B
X-Request-Id=5H4AYXsA7rp
Content-Security-Policy=block-all-mixed-content
Vary=Origin
X-Xss-Protection=1; mode=block
Date=Tue, 26 Aug 2025 04:59:33 GMT
Accept-Ranges=bytes
Content-Length=423
Content-Type=application/xml
target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-26 05:01:49 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>InternalError</Code><Message>We encountered an internal error, please try again.: cause(uplink: metaclient: accounting backend: Redis eval failed: dial tcp 10.37.0.83:6379: operation was canceled)</Message><Key>hello-world.txt</Key><BucketName>uptime-checks</BucketName><Resource>/hello-world.txt</Resource><RequestId>185F3861EA4B077B</RequestId><HostId></HostId></Error>"
Duration: 8.117567295s edge gateway vhost bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
802.494847ms
FAIL (2025-08-26 05:01:32 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F381EFB1C9300, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 802.50697ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..................................................X.....X...]
duration:
3.553129491s
FAIL (2025-08-26 05:01:26 UTC)
orbiter warning: 
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:58:54.632991348 (12h55m9.858174394s) [upload=11349]: Starting upload
Aug 26 04:58:54.633055548 (12h55m9.858238594s) [upload=11349]: Done starting upload
Aug 26 04:58:54.633126050 (12h55m9.858309107s) [upload=11349]: Uploading segments...
Aug 26 04:58:54.633283045 (12h55m9.858466102s) [upload=11349]: Got next segment. Inline: false
Aug 26 04:58:54.633316488 (12h55m9.858499545s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:58:54.633321207 (12h55m9.858504264s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:58:58.655965649 (12h55m13.881148726s) [seg_pos={0 0}, upload=11349]: Begin upload segment...
Aug 26 04:58:58.656054146 (12h55m13.881237223s) [seg_pos={0 0}, upload=11349]: Done begin upload segment.
Aug 26 04:58:58.656098769 (12h55m13.881281847s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656148543 (12h55m13.881331611s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656574924 (12h55m13.881757991s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656741646 (12h55m13.881924724s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656978031 (12h55m13.882161108s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657326154 (12h55m13.882509232s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657343688 (12h55m13.882526764s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657744651 (12h55m13.882927788s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657800345 (12h55m13.882983422s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658144241 (12h55m13.883327318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658262203 (12h55m13.883445279s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658527862 (12h55m13.883710939s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658722727 (12h55m13.883905804s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658883790 (12h55m13.884066867s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659220493 (12h55m13.884403569s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659753644 (12h55m13.884936711s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.754621259 (12h55m13.979804306s) [upload=11349]: Next returned nil segment
Aug 26 04:58:58.754644293 (12h55m13.97982734s) [upload=11349]: Waiting for error group managing segments...
Aug 26 04:58:58.763291261 (12h55m13.988474318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764054023 (12h55m13.98923707s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764064784 (12h55m13.989247841s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765165831 (12h55m13.990348879s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765174608 (12h55m13.990357665s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765575150 (12h55m13.990758197s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765615426 (12h55m13.990798483s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765940345 (12h55m13.991123402s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766080359 (12h55m13.991263406s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766966263 (12h55m13.992149309s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.768385056 (12h55m13.993568093s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.769690068 (12h55m13.994873115s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772653423 (12h55m13.99783646s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772672168 (12h55m13.997855215s) [seg_pos={0 0}, upload=11349]: Segment reached optimal threshold of 13 pieces.
Aug 26 04:58:58.772774941 (12h55m13.997957988s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772793706 (12h55m13.997976753s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772839091 (12h55m13.998022138s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772880569 (12h55m13.998063606s) [seg_pos={0 0}, upload=11349]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 04:58:58.772940883 (12h55m13.99812392s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:58:58.772944299 (12h55m13.998127347s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:06.816180513 (12h55m22.041363581s) [upload=11349]: Done uploading segments...
Aug 26 04:59:06.816202064 (12h55m22.041385141s) [upload=11349]: Upload finished. err: <nil>
Duration: 3.55315041s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
2.165687859s
FAIL (2025-08-26 05:01:25 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
n
Aug 26 04:59:54.491876868 (12h56m9.717059954s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:59:55.898473672 (12h56m11.123656729s) [seg_pos={0 0}, upload=11365]: Begin upload segment...
Aug 26 04:59:55.898578450 (12h56m11.123761507s) [seg_pos={0 0}, upload=11365]: Done begin upload segment.
Aug 26 04:59:55.898604639 (12h56m11.123787696s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898647179 (12h56m11.123830226s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898996365 (12h56m11.124179412s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899235543 (12h56m11.1244186s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899248037 (12h56m11.124431094s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899506833 (12h56m11.12468988s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899574300 (12h56m11.124757347s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899843586 (12h56m11.125026643s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899963090 (12h56m11.125146147s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900138228 (12h56m11.125321285s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900431890 (12h56m11.125614947s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900558217 (12h56m11.125741264s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900861106 (12h56m11.126044163s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900871085 (12h56m11.126054142s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901238495 (12h56m11.126421542s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901672420 (12h56m11.126855466s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.902022848 (12h56m11.127205905s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.903391288 (12h56m11.128574334s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.994352287 (12h56m11.219535334s) [upload=11365]: Next returned nil segment
Aug 26 04:59:55.994374639 (12h56m11.219557687s) [upload=11365]: Waiting for error group managing segments...
Aug 26 04:59:56.001240734 (12h56m11.226423791s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.002465303 (12h56m11.227648349s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003141202 (12h56m11.228324259s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003546413 (12h56m11.22872948s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003630461 (12h56m11.228813518s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004292123 (12h56m11.22947517s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004703165 (12h56m11.229886222s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.006769947 (12h56m11.231952993s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.007034093 (12h56m11.23221715s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.008777136 (12h56m11.233960183s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.011187783 (12h56m11.236370821s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.017859823 (12h56m11.24304287s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027329847 (12h56m11.252512894s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027997051 (12h56m11.253180087s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.028028038 (12h56m11.253211096s) [seg_pos={0 0}, upload=11365]: Segment reached optimal threshold of 14 pieces.
Aug 26 04:59:56.028506516 (12h56m11.253689574s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ofwUBJwLG7eUYo7JE1ufwxRRyQ9HUpTZnshuTz6YF2JMfzVcX): protocol: rpc: dial tcp 79.127.205.245:4249: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028671135 (12h56m11.253854192s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ea2CWMvyMotcFqC6Q1vf2R5UWWZFgwPcmSdkSqMAwCQiVHdoM): protocol: read tcp 5.161.89.252:33180->79.127.205.234:4614: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028792494 (12h56m11.253975541s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XmGcLqbv3ctCx52m42uKqwKpbWjgzEbvoJmdRLXt81qiGvY6M): protocol: read tcp 5.161.89.252:41050->64.49.13.170:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028849591 (12h56m11.254032638s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: read tcp 5.161.89.252:26264->79.127.201.215:4349: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028882292 (12h56m11.25406534s) [seg_pos={0 0}, upload=11365]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 26 04:59:56.028946543 (12h56m11.25412959s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:59:56.028951021 (12h56m11.254134069s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:56.240070251 (12h56m11.465253308s) [upload=11365]: Done uploading segments...
Aug 26 04:59:56.240108883 (12h56m11.46529194s) [upload=11365]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 2.165707056s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [...................................................X....XX..]
duration:
618.51713ms
FAIL (2025-08-26 05:01:25 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F386242827DE1, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 618.536706ms edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...................................................X.....X..]
duration:
23.024000424s
FAIL (2025-08-26 05:01:14 UTC)
orbiter warning: 
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:58:54.632991348 (12h55m9.858174394s) [upload=11349]: Starting upload
Aug 26 04:58:54.633055548 (12h55m9.858238594s) [upload=11349]: Done starting upload
Aug 26 04:58:54.633126050 (12h55m9.858309107s) [upload=11349]: Uploading segments...
Aug 26 04:58:54.633283045 (12h55m9.858466102s) [upload=11349]: Got next segment. Inline: false
Aug 26 04:58:54.633316488 (12h55m9.858499545s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:58:54.633321207 (12h55m9.858504264s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:58:58.655965649 (12h55m13.881148726s) [seg_pos={0 0}, upload=11349]: Begin upload segment...
Aug 26 04:58:58.656054146 (12h55m13.881237223s) [seg_pos={0 0}, upload=11349]: Done begin upload segment.
Aug 26 04:58:58.656098769 (12h55m13.881281847s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656148543 (12h55m13.881331611s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656574924 (12h55m13.881757991s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656741646 (12h55m13.881924724s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.656978031 (12h55m13.882161108s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657326154 (12h55m13.882509232s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657343688 (12h55m13.882526764s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657744651 (12h55m13.882927788s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.657800345 (12h55m13.882983422s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658144241 (12h55m13.883327318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658262203 (12h55m13.883445279s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658527862 (12h55m13.883710939s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658722727 (12h55m13.883905804s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.658883790 (12h55m13.884066867s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659220493 (12h55m13.884403569s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.659753644 (12h55m13.884936711s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Uploading piece...
Aug 26 04:58:58.754621259 (12h55m13.979804306s) [upload=11349]: Next returned nil segment
Aug 26 04:58:58.754644293 (12h55m13.97982734s) [upload=11349]: Waiting for error group managing segments...
Aug 26 04:58:58.763291261 (12h55m13.988474318s) [noise=true , address=45.91.116.6:4973     , piece_id=2T7N24M5AZJOFSEIE4CE3XHQG4IPVM7V4K6UB23QFSLLCGQBS2WA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764054023 (12h55m13.98923707s) [noise=true , address=45.91.116.7:4009     , piece_id=BNHLG72C3N5RIPVUSTYGJ35A5VY4V7C2TVEETPJHL7XXTU7YVJNQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.764064784 (12h55m13.989247841s) [noise=true , address=45.91.116.4:4940     , piece_id=FUGBZELSLQWA53VZ277IDVSVYINOE6UERUDPZ4E7JPJ3TV2SPMZA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765165831 (12h55m13.990348879s) [noise=true , address=45.91.116.10:4082    , piece_id=ZRSS3O6P3R6L67FYOQMFJUGJPF3TSG45LLIEUYTHCVIEQVXSOHLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765174608 (12h55m13.990357665s) [noise=true , address=45.91.116.12:4225    , piece_id=YTCN67WJWS5KGSH632GRCWHRDXE2WDI4JJMJZR2L34LOD7XL2BCA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765575150 (12h55m13.990758197s) [noise=true , address=45.91.116.5:4299     , piece_id=DZJHLFZ7WQ7ITKVTZCR6OLHD3NOVTCMG7W4OGFZWPOA42WDO3R6Q, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765615426 (12h55m13.990798483s) [noise=true , address=45.91.116.14:4191    , piece_id=LDB375D3SOGRLKXGMFXYWYHV36PULDODOYODIC7EXZ7WQ5YH35QA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.765940345 (12h55m13.991123402s) [noise=true , address=77.90.31.14:4663     , piece_id=GZJD4FG7PKCZKROS7KLJKUZSKC4SBNARBAMJCSDGPKT3ACLUO5XA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766080359 (12h55m13.991263406s) [noise=true , address=77.90.31.21:4877     , piece_id=AIQKYZTWYE3UTVHUIBPECBM6RELDMRFQFEKTWLQSRHS2XALNXPCQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.766966263 (12h55m13.992149309s) [noise=true , address=104.36.82.11:4113    , piece_id=SDGBG43E3HUKSKMEVNNFV6JCXK6ZL2QITBFK2YOM5454XWZXMOLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.768385056 (12h55m13.993568093s) [noise=true , address=104.36.82.2:4729     , piece_id=3QF3YPMORGE4WFHUASNK37IJTQIX5XCJ7Q4QV4BRQIAVQOOWOGUA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.769690068 (12h55m13.994873115s) [noise=true , address=104.36.82.12:4150    , piece_id=6LHUEVSPM325VIYHBNQB2QR33OITE3RYAILPZHNYYFZNXLDL3RIQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772653423 (12h55m13.99783646s) [noise=true , address=104.36.82.4:4803     , piece_id=JFRHTFEJNCAHR3XXYIYQZFPYR6TFCX3USA4FAHJCYJSW45IQ3IRQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: <nil>
Aug 26 04:58:58.772672168 (12h55m13.997855215s) [seg_pos={0 0}, upload=11349]: Segment reached optimal threshold of 13 pieces.
Aug 26 04:58:58.772774941 (12h55m13.997957988s) [noise=true , address=104.36.82.12:4149    , piece_id=Y7V3LV2AUMXLAPXBN2FAUSVD32CHUVYFEQ2OQIJNP2WQVAFPV3YA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772793706 (12h55m13.997976753s) [noise=true , address=104.36.82.16:4295    , piece_id=3VUJ3LMF6FNF5EZKFKJFQIVAHRKNG3NN7C2STSRXF4WSUVIR4FLQ, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772839091 (12h55m13.998022138s) [noise=true , address=104.36.82.9:4005     , piece_id=VYYJWYYJAVFYV2LEV7JV2F7SWC7HAEXVIVH5ELD457574CGSJMVA, seg_pos={0 0}, upload=11349]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: expected piece hash; context canceled; EOF
Aug 26 04:58:58.772880569 (12h55m13.998063606s) [seg_pos={0 0}, upload=11349]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Aug 26 04:58:58.772940883 (12h55m13.99812392s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:58:58.772944299 (12h55m13.998127347s) [upload=11349]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:06.816180513 (12h55m22.041363581s) [upload=11349]: Done uploading segments...
Aug 26 04:59:06.816202064 (12h55m22.041385141s) [upload=11349]: Upload finished. err: <nil>
Duration: 23.024018948s core uplink us-select-1
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.us1.linksharing)
antiblip:
1 out of 60 is failed (threshold=6) [.........................................................X..]
duration:
276.955907ms
remote:
"tcp:136.0.77.2:443"
response headers:
Date=Tue, 26 Aug 2025 04:59:41 GMT
Access-Control-Allow-Headers=*
Access-Control-Allow-Methods=GET, HEAD
Access-Control-Allow-Origin=*
Content-Type=text/html; charset=utf-8
Accept-Ranges=bytes
X-Request-Id=YDWvXrmwXNN
target:
https://link.storjshare.io/raw/jw7f74fzbtwblz3mwqolmnivikxq/uptime-checks/hello-world.txt
FAIL (2025-08-26 05:01:14 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"<!DOCTYPE html>\n<html lang=\"en\">\n\n<head>\n  <meta charset=\"utf-8\">\n  <title>Error | Storj</title>\n\n  <meta property=\"og:url\" content=\"https://link.storjshare.io\" />\n  <meta property=\"og:title\" content=\"Error | Storj\" />\n  <meta property=\"og:description\" name=\"description\" content=\"Shared content - Storj\" />\n  \n  <meta property=\"og:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <meta name=\"twitter:card\" content=\"summary\" />\n  <meta name=\"twitter:site\" content=\"@storj\" />\n  <meta name=\"twitter:title\" content=\"Error | Storj\" />\n  <meta name=\"twitter:description\" content=\"Shared content - Storj\" />\n  \n  <meta name=\"twitter:image\" content=\"https://link.storjshare.io/static/img/file.png?v=f0d1e95\" />\n  \n\n  <link rel=\"shortcut icon\"\n    href=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAIAAAD8GO2jAAAC40lEQVRIDe1US0iUURQ+V8fJmcnykTJmNA0tMszETUVlCrkoR+2lWdFDk7AHlNQiCGaREhlUli3ETSRFqx4UlAi5ySgqS8hQiXLMTKca35k6j3O6d5zE//rnuNCF4Mdd3HvO+c53/3P/cxgRwUwiY"
Duration: 276.980233ms edge linksharing bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
3 out of 60 is failed (threshold=3) [................................................XX.......X..]
duration:
29.776141366s
FAIL (2025-08-26 05:01:13 UTC)
operation: Commit
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 29.7761696s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
17.365637516s
FAIL (2025-08-26 05:01:13 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
======LOGS======
n
Aug 26 04:59:54.491876868 (12h56m9.717059954s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 26 04:59:55.898473672 (12h56m11.123656729s) [seg_pos={0 0}, upload=11365]: Begin upload segment...
Aug 26 04:59:55.898578450 (12h56m11.123761507s) [seg_pos={0 0}, upload=11365]: Done begin upload segment.
Aug 26 04:59:55.898604639 (12h56m11.123787696s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898647179 (12h56m11.123830226s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.898996365 (12h56m11.124179412s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899235543 (12h56m11.1244186s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899248037 (12h56m11.124431094s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899506833 (12h56m11.12468988s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899574300 (12h56m11.124757347s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899843586 (12h56m11.125026643s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.899963090 (12h56m11.125146147s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900138228 (12h56m11.125321285s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900431890 (12h56m11.125614947s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900558217 (12h56m11.125741264s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900861106 (12h56m11.126044163s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.900871085 (12h56m11.126054142s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901238495 (12h56m11.126421542s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.901672420 (12h56m11.126855466s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.902022848 (12h56m11.127205905s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.903391288 (12h56m11.128574334s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Uploading piece...
Aug 26 04:59:55.994352287 (12h56m11.219535334s) [upload=11365]: Next returned nil segment
Aug 26 04:59:55.994374639 (12h56m11.219557687s) [upload=11365]: Waiting for error group managing segments...
Aug 26 04:59:56.001240734 (12h56m11.226423791s) [noise=true , address=77.90.31.21:4894     , piece_id=3L6GHQRVJKQWJODDAX2OQG7GTFVPJRD7YUB74WWHRUDMG4R75UEA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.002465303 (12h56m11.227648349s) [noise=true , address=79.127.219.51:4189   , piece_id=5JSQRA7N4OZHBU36WFUAXOSRTPFWU7GXNDSXAQ2C2AJH2TBZ33AQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003141202 (12h56m11.228324259s) [noise=true , address=45.91.116.4:4962     , piece_id=Q6ID4T4YXQYLSNJRNMRFN5KM5TUNNBECBMFSILKT7CZMDD3SIH2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003546413 (12h56m11.22872948s) [noise=true , address=77.90.31.13:4648     , piece_id=L3P6DIWH2NUBBWSI4IJO5IPMA4PV2RO7CE3SHQSZVZFOTQUDNWMA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.003630461 (12h56m11.228813518s) [noise=true , address=45.91.116.9:4277     , piece_id=V37TBD24DM4BYF6MTFQ5HNTOV3XCPBTMCT5IW5WOCI66ZLW3HGLA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004292123 (12h56m11.22947517s) [noise=true , address=77.90.31.19:4858     , piece_id=K5NZT5JLGVGPMGFBGQ4QLFAFVXWTBBQIENZH6AOCTCGS4K73HVUQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.004703165 (12h56m11.229886222s) [noise=true , address=45.91.116.12:4236    , piece_id=WHBR5GGVR4DGSUTNYLHR34G23UIBN6S3L3XIVABZB5EB47SKQOYQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.006769947 (12h56m11.231952993s) [noise=true , address=77.90.31.16:4753     , piece_id=3NNGQFADR6O6ZRYBZRREAC65PYLAYAYX2TSBJJDM6H6ALOME7M4Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.007034093 (12h56m11.23221715s) [noise=true , address=77.90.31.10:4540     , piece_id=NOI3C7DCNHBQQFQ2FOQWM62O4NX4FYCPUQQMTJQSINOQSBGYEJ2Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.008777136 (12h56m11.233960183s) [noise=true , address=104.36.82.60:4359    , piece_id=ZVAY4BNJTCRSLHIU7TJL6YFRFGDCY4KBADLW2M5NAEUDTMANGM5A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.011187783 (12h56m11.236370821s) [noise=true , address=104.36.82.14:4250    , piece_id=IZDGBS5SR3ZULR44LW5RB5FAQPAXZ4MQTK7I35HZ6NV3AESRDC4A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.017859823 (12h56m11.24304287s) [noise=true , address=77.90.31.12:4612     , piece_id=75CREFPJH2PDJCE2DUMNKAAVUKW6FVP3KEDXBIL4DBMRWSLQS4SA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027329847 (12h56m11.252512894s) [noise=true , address=104.36.82.5:4856     , piece_id=6YB2UEERGWMZEWGNH44WPNTLE56POWRAOPUHXBIBHSDVZBQ3ZAHA, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.027997051 (12h56m11.253180087s) [noise=true , address=104.36.82.7:4913     , piece_id=H4MIIKMVPHAB7YSYHQSPDF5IXZYBNDEPOYJJDYGE56QO4QH5NMDQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: <nil>
Aug 26 04:59:56.028028038 (12h56m11.253211096s) [seg_pos={0 0}, upload=11365]: Segment reached optimal threshold of 14 pieces.
Aug 26 04:59:56.028506516 (12h56m11.253689574s) [noise=true , address=79.127.205.245:4249  , piece_id=N2MFPJKX3FH77ROEMJDKNDC3VKTEGFV55UEGF26M67FF5ZM3627A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ofwUBJwLG7eUYo7JE1ufwxRRyQ9HUpTZnshuTz6YF2JMfzVcX): protocol: rpc: dial tcp 79.127.205.245:4249: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028671135 (12h56m11.253854192s) [noise=true , address=79.127.205.234:4614  , piece_id=P3GLUAQDNCB5NQQX5CE5BYLDO6OSGRHJZTISFFTY3DQAZG3K6Q7Q, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ea2CWMvyMotcFqC6Q1vf2R5UWWZFgwPcmSdkSqMAwCQiVHdoM): protocol: read tcp 5.161.89.252:33180->79.127.205.234:4614: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028792494 (12h56m11.253975541s) [noise=true , address=64.49.13.170:4728    , piece_id=HWNSOOHT7UM6UQGAK7UBTCTMODWAY4LDKS5GIM3F363FU55VTNFQ, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XmGcLqbv3ctCx52m42uKqwKpbWjgzEbvoJmdRLXt81qiGvY6M): protocol: read tcp 5.161.89.252:41050->64.49.13.170:4728: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028849591 (12h56m11.254032638s) [noise=true , address=79.127.201.215:4349  , piece_id=UNQBQ4NE2SUH7OBPLAPYJSBZ3KNLEPTNBU6E457WHYNRFF3KKM3A, seg_pos={0 0}, upload=11365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1y1KYQJsvbNxfySU7mXySUmJtEDVAE7fst1XVRHU5N8Txmjn5T): protocol: read tcp 5.161.89.252:26264->79.127.201.215:4349: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 26 04:59:56.028882292 (12h56m11.25406534s) [seg_pos={0 0}, upload=11365]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 26 04:59:56.028946543 (12h56m11.25412959s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 26 04:59:56.028951021 (12h56m11.254134069s) [upload=11365]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:56.240070251 (12h56m11.465253308s) [upload=11365]: Done uploading segments...
Aug 26 04:59:56.240108883 (12h56m11.46529194s) [upload=11365]: Upload finished. err: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 17.365655049s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [.................................................X.....XX...]
duration:
979.017438ms
FAIL (2025-08-26 05:01:13 UTC)
operation: Commit
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 979.038337ms core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [................................................XX.......X..]
duration:
7.536886102s
FAIL (2025-08-26 05:01:12 UTC)
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
Aug 26 04:59:12.156304404 (12h55m27.381487481s) [upload=11354]: Starting upload
Aug 26 04:59:12.156450137 (12h55m27.381633214s) [upload=11354]: Done starting upload
Aug 26 04:59:12.156550065 (12h55m27.381733152s) [upload=11354]: Uploading segments...
Aug 26 04:59:12.156613884 (12h55m27.381796961s) [upload=11354]: Got next segment. Inline: true
Aug 26 04:59:12.156623683 (12h55m27.381806759s) [upload=11354]: Waiting for error group managing segments...
Aug 26 04:59:12.156689306 (12h55m27.381872394s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 26 04:59:12.156697092 (12h55m27.381880168s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 26 04:59:12.156703453 (12h55m27.381886561s) [upload=11354]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 26 04:59:42.137624713 (12h55m57.36280777s) [upload=11354]: Done uploading segments...
Aug 26 04:59:42.137641224 (12h55m57.362824281s) [upload=11354]: Upload finished. err: metaclient: context deadline exceeded
Duration: 7.53690667s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)
antiblip:
3 out of 60 is failed (threshold=3) [..................................................X.....XX..]
duration:
29.405964406s
FAIL (2025-08-26 05:01:12 UTC)
operation: Commit
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 29.405984624s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
29.498587104s
FAIL (2025-08-26 05:01:12 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 429, RequestID: 185F381EFB1C9300, HostID: , api error SlowDown: Please reduce your request rate.
Duration: 29.498610829s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)
antiblip:
3 out of 60 is failed (threshold=3) [.................................................XX.......X.]
duration:
15.837901262s
FAIL (2025-08-26 05:00:42 UTC)
operation: Commit
uplink: metaclient: accounting backend: Redis pipeline exec failed: dial tcp 10.37.0.83:6379: operation was canceled
Duration: 15.837920429s core uplink us1