Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 117669 reports (Page 1540 of 2354)
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
8 out of 15 is failed (threshold=6) [X...XX..XXXX..X]
duration:
1.042188945s
remote:
"tcp:34.202.203.47:443"
response headers:
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:38:33 GMT
Content-Type=text/html
target:
https://www.storj.io/
FAIL (2025-08-21 17:38:33 UTC)
unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.042215054s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration:
310.185431ms
FAIL (2025-08-21 17:38:24 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 310.284067ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
8 out of 15 is failed (threshold=6) [XX...XX..XXXX..]
duration:
85.997287ms
remote:
"tcp:34.202.203.47:443"
response headers:
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:36:18 GMT
target:
https://www.storj.io/
FAIL (2025-08-21 17:38:13 UTC)
unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 86.023897ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
130.660557ms
FAIL (2025-08-21 17:38:08 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 130.686374ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
252.491857ms
FAIL (2025-08-21 17:37:48 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 252.51458ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.........X............................................X.....]
duration:
2.2948712s
FAIL (2025-08-21 17:37:41 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:33:59.350385001 (312h7m40.081652621s) [upload=324499]: Starting upload
Aug 21 17:33:59.350415309 (312h7m40.08168294s) [upload=324499]: Done starting upload
Aug 21 17:33:59.350640743 (312h7m40.081908373s) [upload=324499]: Uploading segments...
Aug 21 17:33:59.351323566 (312h7m40.082591187s) [upload=324499]: Got next segment. Inline: false
Aug 21 17:33:59.351374332 (312h7m40.082641952s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:33:59.351378549 (312h7m40.08264617s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 17:33:59.354517922 (312h7m40.085785582s) [upload=324499]: Next segment err: metaclient: Too Many Requests
Aug 21 17:33:59.354532639 (312h7m40.085800269s) [upload=324499]: Done uploading segments...
Aug 21 17:33:59.354542808 (312h7m40.085810438s) [upload=324499]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.294906735s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
5 out of 15 is failed (threshold=6) [...X....XXXX...]
duration:
91.770108ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Thu, 21 Aug 2025 17:35:05 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:37:32 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 91.791065ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
9 out of 15 is failed (threshold=6) [XXX...XX..XXXX.]
duration:
145.877208ms
remote:
"tcp:34.202.203.47:443"
response headers:
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:36:18 GMT
target:
https://www.storj.io/
FAIL (2025-08-21 17:37:20 UTC)
unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 145.895012ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..........X............................................X....]
duration:
404.484023ms
FAIL (2025-08-21 17:37:15 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:33:59.350385001 (312h7m40.081652621s) [upload=324499]: Starting upload
Aug 21 17:33:59.350415309 (312h7m40.08168294s) [upload=324499]: Done starting upload
Aug 21 17:33:59.350640743 (312h7m40.081908373s) [upload=324499]: Uploading segments...
Aug 21 17:33:59.351323566 (312h7m40.082591187s) [upload=324499]: Got next segment. Inline: false
Aug 21 17:33:59.351374332 (312h7m40.082641952s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:33:59.351378549 (312h7m40.08264617s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 17:33:59.354517922 (312h7m40.085785582s) [upload=324499]: Next segment err: metaclient: Too Many Requests
Aug 21 17:33:59.354532639 (312h7m40.085800269s) [upload=324499]: Done uploading segments...
Aug 21 17:33:59.354542808 (312h7m40.085810438s) [upload=324499]: Upload finished. err: metaclient: Too Many Requests
Duration: 404.507567ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
342.049676ms
FAIL (2025-08-21 17:36:56 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 342.082108ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
5 out of 15 is failed (threshold=6) [....X....XXXX..]
duration:
91.75979ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Thu, 21 Aug 2025 17:35:05 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:36:35 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 91.78122ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
9 out of 15 is failed (threshold=6) [.XXX...XX..XXXX]
duration:
1.23171305s
remote:
"tcp:34.202.203.47:443"
response headers:
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:36:18 GMT
target:
https://www.storj.io/
FAIL (2025-08-21 17:36:18 UTC)
unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.231746924s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...........X............................................X...]
duration:
1.175884745s
FAIL (2025-08-21 17:36:15 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:33:59.350385001 (312h7m40.081652621s) [upload=324499]: Starting upload
Aug 21 17:33:59.350415309 (312h7m40.08168294s) [upload=324499]: Done starting upload
Aug 21 17:33:59.350640743 (312h7m40.081908373s) [upload=324499]: Uploading segments...
Aug 21 17:33:59.351323566 (312h7m40.082591187s) [upload=324499]: Got next segment. Inline: false
Aug 21 17:33:59.351374332 (312h7m40.082641952s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:33:59.351378549 (312h7m40.08264617s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 17:33:59.354517922 (312h7m40.085785582s) [upload=324499]: Next segment err: metaclient: Too Many Requests
Aug 21 17:33:59.354532639 (312h7m40.085800269s) [upload=324499]: Done uploading segments...
Aug 21 17:33:59.354542808 (312h7m40.085810438s) [upload=324499]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.175911135s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
127.553061ms
FAIL (2025-08-21 17:35:59 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 127.577036ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
5 out of 15 is failed (threshold=6) [.....X....XXXX.]
duration:
421.787555ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Thu, 21 Aug 2025 17:35:05 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:35:45 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 421.813514ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
8 out of 15 is failed (threshold=6) [..XXX...XX..XXX]
duration:
1.102549303s
remote:
"tcp:34.202.203.47:443"
response headers:
Date=Thu, 21 Aug 2025 17:35:37 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/
FAIL (2025-08-21 17:35:37 UTC)
unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.102568239s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [............X............................................X..]
duration:
405.307465ms
FAIL (2025-08-21 17:35:21 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:33:59.350385001 (312h7m40.081652621s) [upload=324499]: Starting upload
Aug 21 17:33:59.350415309 (312h7m40.08168294s) [upload=324499]: Done starting upload
Aug 21 17:33:59.350640743 (312h7m40.081908373s) [upload=324499]: Uploading segments...
Aug 21 17:33:59.351323566 (312h7m40.082591187s) [upload=324499]: Got next segment. Inline: false
Aug 21 17:33:59.351374332 (312h7m40.082641952s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:33:59.351378549 (312h7m40.08264617s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 17:33:59.354517922 (312h7m40.085785582s) [upload=324499]: Next segment err: metaclient: Too Many Requests
Aug 21 17:33:59.354532639 (312h7m40.085800269s) [upload=324499]: Done uploading segments...
Aug 21 17:33:59.354542808 (312h7m40.085810438s) [upload=324499]: Upload finished. err: metaclient: Too Many Requests
Duration: 405.35216ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
7 out of 15 is failed (threshold=6) [...XXX...XX..XX]
duration:
1.021381797s
remote:
"tcp:54.243.86.28:443"
response headers:
Date=Thu, 21 Aug 2025 17:35:18 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/
FAIL (2025-08-21 17:35:18 UTC)
unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.021407175s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
157.830924ms
FAIL (2025-08-21 17:35:13 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 157.857053ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
5 out of 15 is failed (threshold=6) [......X....XXXX]
duration:
1.023542549s
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Thu, 21 Aug 2025 17:35:05 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:35:05 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.023554902s core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
6 out of 15 is failed (threshold=6) [....XXX...XX..X]
duration:
1.011786903s
remote:
"tcp:34.202.203.47:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:34:59 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-21 17:34:59 UTC)
unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.011808824s core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
5 out of 15 is failed (threshold=6) [X......X....XXX]
duration:
1.019491719s
remote:
"tcp:18.211.166.153:443"
response headers:
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:34:37 GMT
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:34:37 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.019527369s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.............X............................................X.]
duration:
2.584508507s
FAIL (2025-08-21 17:34:29 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:33:59.350385001 (312h7m40.081652621s) [upload=324499]: Starting upload
Aug 21 17:33:59.350415309 (312h7m40.08168294s) [upload=324499]: Done starting upload
Aug 21 17:33:59.350640743 (312h7m40.081908373s) [upload=324499]: Uploading segments...
Aug 21 17:33:59.351323566 (312h7m40.082591187s) [upload=324499]: Got next segment. Inline: false
Aug 21 17:33:59.351374332 (312h7m40.082641952s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:33:59.351378549 (312h7m40.08264617s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 17:33:59.354517922 (312h7m40.085785582s) [upload=324499]: Next segment err: metaclient: Too Many Requests
Aug 21 17:33:59.354532639 (312h7m40.085800269s) [upload=324499]: Done uploading segments...
Aug 21 17:33:59.354542808 (312h7m40.085810438s) [upload=324499]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.584536199s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
168.767225ms
FAIL (2025-08-21 17:34:29 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 168.785288ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
5 out of 15 is failed (threshold=6) [.....XXX...XX..]
duration:
370.929732ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Thu, 21 Aug 2025 17:32:22 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/
FAIL (2025-08-21 17:34:09 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 370.948446ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
4 out of 15 is failed (threshold=6) [.X......X....XX]
duration:
1.023848449s
remote:
"tcp:18.211.166.153:443"
response headers:
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:34:08 GMT
Content-Type=text/html
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:34:08 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.023877523s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
365.300882ms
FAIL (2025-08-21 17:34:06 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 365.32626ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..............X............................................X]
duration:
18.613473ms
FAIL (2025-08-21 17:33:59 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:33:59.350385001 (312h7m40.081652621s) [upload=324499]: Starting upload
Aug 21 17:33:59.350415309 (312h7m40.08168294s) [upload=324499]: Done starting upload
Aug 21 17:33:59.350640743 (312h7m40.081908373s) [upload=324499]: Uploading segments...
Aug 21 17:33:59.351323566 (312h7m40.082591187s) [upload=324499]: Got next segment. Inline: false
Aug 21 17:33:59.351374332 (312h7m40.082641952s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:33:59.351378549 (312h7m40.08264617s) [upload=324499]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 17:33:59.354517922 (312h7m40.085785582s) [upload=324499]: Next segment err: metaclient: Too Many Requests
Aug 21 17:33:59.354532639 (312h7m40.085800269s) [upload=324499]: Done uploading segments...
Aug 21 17:33:59.354542808 (312h7m40.085810438s) [upload=324499]: Upload finished. err: metaclient: Too Many Requests
Duration: 18.631157ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
5 out of 15 is failed (threshold=6) [......XXX...XX.]
duration:
112.64895ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Thu, 21 Aug 2025 17:32:22 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/
FAIL (2025-08-21 17:33:14 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 112.667867ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
3 out of 15 is failed (threshold=6) [..X......X....X]
duration:
1.025426891s
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:33:10 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:33:10 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.025454983s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................]
duration:
1.27559604s
FAIL (2025-08-21 17:32:58 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: invalid range 0:4194304 (size:4194304)
======LOGS======
Aug 21 16:58:11.097765980 (311h31m51.82903362s) [upload=323881]: Starting upload
Aug 21 16:58:11.097816425 (311h31m51.829084066s) [upload=323881]: Done starting upload
Aug 21 16:58:11.097974212 (311h31m51.829241862s) [upload=323881]: Uploading segments...
Aug 21 16:58:11.098124976 (311h31m51.829392606s) [upload=323881]: Got next segment. Inline: false
Aug 21 16:58:11.098190559 (311h31m51.829458199s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 16:58:11.098198244 (311h31m51.829465904s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 16:58:11.186959668 (311h31m51.918227288s) [seg_pos={0 0}, upload=323881]: Begin upload segment...
Aug 21 16:58:11.187010914 (311h31m51.918278574s) [seg_pos={0 0}, upload=323881]: Done begin upload segment.
Aug 21 16:58:11.187041721 (311h31m51.918309351s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187059065 (311h31m51.918326695s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187096966 (311h31m51.918364587s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187499954 (311h31m51.918767574s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187516114 (311h31m51.918783745s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187870982 (311h31m51.919138612s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187878215 (311h31m51.919145845s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188266736 (311h31m51.919534356s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188291242 (311h31m51.919558873s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188643465 (311h31m51.919911095s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188658473 (311h31m51.919926103s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188993774 (311h31m51.920261394s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189017548 (311h31m51.920285178s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189365152 (311h31m51.920632782s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189952657 (311h31m51.921220287s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.190704210 (311h31m51.92197184s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.191563527 (311h31m51.922831157s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.192590128 (311h31m51.923857758s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.292199435 (311h31m52.023467055s) [upload=323881]: Next returned nil segment
Aug 21 16:58:11.292208382 (311h31m52.023476002s) [upload=323881]: Waiting for error group managing segments...
Aug 21 16:58:11.305981536 (311h31m52.037249156s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.317455457 (311h31m52.048723076s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.319650365 (311h31m52.050917985s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.320890678 (311h31m52.052158297s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.325965804 (311h31m52.057233424s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.327860037 (311h31m52.059127667s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.328132208 (311h31m52.059399828s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.330010401 (311h31m52.061278021s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.335413914 (311h31m52.066681533s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.341151935 (311h31m52.072419546s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.348847890 (311h31m52.08011552s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.360184613 (311h31m52.091452233s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.366983571 (311h31m52.098251201s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392521653 (311h31m52.123789274s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392540769 (311h31m52.123808399s) [seg_pos={0 0}, upload=323881]: Segment reached optimal threshold of 14 pieces.
Aug 21 16:58:11.392827879 (311h31m52.124095509s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XaE29W27ym2RmuGdBvHeKcQZZEUbask1zsm9Bz8xLokZNGfcj): protocol: expected piece hash; context canceled; EOF
Aug 21 16:58:11.393068121 (311h31m52.124335742s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1h1gxgVjoHUkjKXA5xhTowGKi9q6wZGyzg8eqKgMHqsxF1ut5P): protocol: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection
Aug 21 16:58:11.393129707 (311h31m52.124397326s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:61860->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 21 16:58:11.393280721 (311h31m52.124548371s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.393342206 (311h31m52.124609846s) [seg_pos={0 0}, upload=323881]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 21 16:58:11.393461841 (311h31m52.124729492s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 21 16:58:11.393467823 (311h31m52.124735463s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 16:58:11.446017719 (311h31m52.177285349s) [upload=323881]: Done uploading segments...
Aug 21 16:58:11.446031615 (311h31m52.177299245s) [upload=323881]: Upload finished. err: <nil>
Duration: 1.275615407s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
546.3745ms
FAIL (2025-08-21 17:32:56 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 546.39575ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
2 out of 15 is failed (threshold=6) [...X......X....]
duration:
80.722769ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:29:19 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:32:23 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 80.747485ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
5 out of 15 is failed (threshold=6) [.......XXX...XX]
duration:
1.014948474s
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Thu, 21 Aug 2025 17:32:22 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/
FAIL (2025-08-21 17:32:22 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.014968231s core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
2 out of 15 is failed (threshold=6) [....X......X...]
duration:
273.270862ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:29:19 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:32:04 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 273.297371ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........]
duration:
153.460738ms
FAIL (2025-08-21 17:31:59 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 153.497116ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
1.439495293s
FAIL (2025-08-21 17:31:48 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: invalid range 0:4194304 (size:4194304)
======LOGS======
Aug 21 16:58:11.097765980 (311h31m51.82903362s) [upload=323881]: Starting upload
Aug 21 16:58:11.097816425 (311h31m51.829084066s) [upload=323881]: Done starting upload
Aug 21 16:58:11.097974212 (311h31m51.829241862s) [upload=323881]: Uploading segments...
Aug 21 16:58:11.098124976 (311h31m51.829392606s) [upload=323881]: Got next segment. Inline: false
Aug 21 16:58:11.098190559 (311h31m51.829458199s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 16:58:11.098198244 (311h31m51.829465904s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 16:58:11.186959668 (311h31m51.918227288s) [seg_pos={0 0}, upload=323881]: Begin upload segment...
Aug 21 16:58:11.187010914 (311h31m51.918278574s) [seg_pos={0 0}, upload=323881]: Done begin upload segment.
Aug 21 16:58:11.187041721 (311h31m51.918309351s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187059065 (311h31m51.918326695s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187096966 (311h31m51.918364587s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187499954 (311h31m51.918767574s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187516114 (311h31m51.918783745s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187870982 (311h31m51.919138612s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187878215 (311h31m51.919145845s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188266736 (311h31m51.919534356s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188291242 (311h31m51.919558873s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188643465 (311h31m51.919911095s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188658473 (311h31m51.919926103s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188993774 (311h31m51.920261394s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189017548 (311h31m51.920285178s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189365152 (311h31m51.920632782s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189952657 (311h31m51.921220287s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.190704210 (311h31m51.92197184s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.191563527 (311h31m51.922831157s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.192590128 (311h31m51.923857758s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.292199435 (311h31m52.023467055s) [upload=323881]: Next returned nil segment
Aug 21 16:58:11.292208382 (311h31m52.023476002s) [upload=323881]: Waiting for error group managing segments...
Aug 21 16:58:11.305981536 (311h31m52.037249156s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.317455457 (311h31m52.048723076s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.319650365 (311h31m52.050917985s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.320890678 (311h31m52.052158297s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.325965804 (311h31m52.057233424s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.327860037 (311h31m52.059127667s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.328132208 (311h31m52.059399828s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.330010401 (311h31m52.061278021s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.335413914 (311h31m52.066681533s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.341151935 (311h31m52.072419546s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.348847890 (311h31m52.08011552s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.360184613 (311h31m52.091452233s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.366983571 (311h31m52.098251201s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392521653 (311h31m52.123789274s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392540769 (311h31m52.123808399s) [seg_pos={0 0}, upload=323881]: Segment reached optimal threshold of 14 pieces.
Aug 21 16:58:11.392827879 (311h31m52.124095509s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XaE29W27ym2RmuGdBvHeKcQZZEUbask1zsm9Bz8xLokZNGfcj): protocol: expected piece hash; context canceled; EOF
Aug 21 16:58:11.393068121 (311h31m52.124335742s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1h1gxgVjoHUkjKXA5xhTowGKi9q6wZGyzg8eqKgMHqsxF1ut5P): protocol: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection
Aug 21 16:58:11.393129707 (311h31m52.124397326s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:61860->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 21 16:58:11.393280721 (311h31m52.124548371s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.393342206 (311h31m52.124609846s) [seg_pos={0 0}, upload=323881]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 21 16:58:11.393461841 (311h31m52.124729492s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 21 16:58:11.393467823 (311h31m52.124735463s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 16:58:11.446017719 (311h31m52.177285349s) [upload=323881]: Done uploading segments...
Aug 21 16:58:11.446031615 (311h31m52.177299245s) [upload=323881]: Upload finished. err: <nil>
Duration: 1.439529458s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......]
duration:
144.46249ms
FAIL (2025-08-21 17:31:39 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 144.479523ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
4 out of 15 is failed (threshold=6) [........XXX...X]
duration:
1.069434348s
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Thu, 21 Aug 2025 17:31:38 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/
FAIL (2025-08-21 17:31:38 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.069457722s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [X................X..........................................]
duration:
1.889875313s
FAIL (2025-08-21 17:31:22 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: invalid range 0:4194304 (size:4194304)
======LOGS======
Aug 21 16:58:11.097765980 (311h31m51.82903362s) [upload=323881]: Starting upload
Aug 21 16:58:11.097816425 (311h31m51.829084066s) [upload=323881]: Done starting upload
Aug 21 16:58:11.097974212 (311h31m51.829241862s) [upload=323881]: Uploading segments...
Aug 21 16:58:11.098124976 (311h31m51.829392606s) [upload=323881]: Got next segment. Inline: false
Aug 21 16:58:11.098190559 (311h31m51.829458199s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 16:58:11.098198244 (311h31m51.829465904s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 16:58:11.186959668 (311h31m51.918227288s) [seg_pos={0 0}, upload=323881]: Begin upload segment...
Aug 21 16:58:11.187010914 (311h31m51.918278574s) [seg_pos={0 0}, upload=323881]: Done begin upload segment.
Aug 21 16:58:11.187041721 (311h31m51.918309351s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187059065 (311h31m51.918326695s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187096966 (311h31m51.918364587s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187499954 (311h31m51.918767574s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187516114 (311h31m51.918783745s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187870982 (311h31m51.919138612s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187878215 (311h31m51.919145845s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188266736 (311h31m51.919534356s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188291242 (311h31m51.919558873s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188643465 (311h31m51.919911095s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188658473 (311h31m51.919926103s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188993774 (311h31m51.920261394s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189017548 (311h31m51.920285178s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189365152 (311h31m51.920632782s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189952657 (311h31m51.921220287s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.190704210 (311h31m51.92197184s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.191563527 (311h31m51.922831157s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.192590128 (311h31m51.923857758s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.292199435 (311h31m52.023467055s) [upload=323881]: Next returned nil segment
Aug 21 16:58:11.292208382 (311h31m52.023476002s) [upload=323881]: Waiting for error group managing segments...
Aug 21 16:58:11.305981536 (311h31m52.037249156s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.317455457 (311h31m52.048723076s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.319650365 (311h31m52.050917985s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.320890678 (311h31m52.052158297s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.325965804 (311h31m52.057233424s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.327860037 (311h31m52.059127667s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.328132208 (311h31m52.059399828s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.330010401 (311h31m52.061278021s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.335413914 (311h31m52.066681533s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.341151935 (311h31m52.072419546s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.348847890 (311h31m52.08011552s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.360184613 (311h31m52.091452233s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.366983571 (311h31m52.098251201s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392521653 (311h31m52.123789274s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392540769 (311h31m52.123808399s) [seg_pos={0 0}, upload=323881]: Segment reached optimal threshold of 14 pieces.
Aug 21 16:58:11.392827879 (311h31m52.124095509s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XaE29W27ym2RmuGdBvHeKcQZZEUbask1zsm9Bz8xLokZNGfcj): protocol: expected piece hash; context canceled; EOF
Aug 21 16:58:11.393068121 (311h31m52.124335742s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1h1gxgVjoHUkjKXA5xhTowGKi9q6wZGyzg8eqKgMHqsxF1ut5P): protocol: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection
Aug 21 16:58:11.393129707 (311h31m52.124397326s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:61860->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 21 16:58:11.393280721 (311h31m52.124548371s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.393342206 (311h31m52.124609846s) [seg_pos={0 0}, upload=323881]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 21 16:58:11.393461841 (311h31m52.124729492s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 21 16:58:11.393467823 (311h31m52.124735463s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 16:58:11.446017719 (311h31m52.177285349s) [upload=323881]: Done uploading segments...
Aug 21 16:58:11.446031615 (311h31m52.177299245s) [upload=323881]: Upload finished. err: <nil>
Duration: 1.889913965s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
284.198485ms
FAIL (2025-08-21 17:31:12 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 284.223062ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
2 out of 15 is failed (threshold=6) [.....X......X..]
duration:
404.017487ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:29:19 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:31:06 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 404.035211ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
3 out of 15 is failed (threshold=6) [.........XXX...]
duration:
64.171274ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:28:40 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-21 17:30:35 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 64.195629ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.X................X.........................................]
duration:
1.908188167s
FAIL (2025-08-21 17:30:30 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: invalid range 0:4194304 (size:4194304)
======LOGS======
Aug 21 16:58:11.097765980 (311h31m51.82903362s) [upload=323881]: Starting upload
Aug 21 16:58:11.097816425 (311h31m51.829084066s) [upload=323881]: Done starting upload
Aug 21 16:58:11.097974212 (311h31m51.829241862s) [upload=323881]: Uploading segments...
Aug 21 16:58:11.098124976 (311h31m51.829392606s) [upload=323881]: Got next segment. Inline: false
Aug 21 16:58:11.098190559 (311h31m51.829458199s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 16:58:11.098198244 (311h31m51.829465904s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 16:58:11.186959668 (311h31m51.918227288s) [seg_pos={0 0}, upload=323881]: Begin upload segment...
Aug 21 16:58:11.187010914 (311h31m51.918278574s) [seg_pos={0 0}, upload=323881]: Done begin upload segment.
Aug 21 16:58:11.187041721 (311h31m51.918309351s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187059065 (311h31m51.918326695s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187096966 (311h31m51.918364587s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187499954 (311h31m51.918767574s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187516114 (311h31m51.918783745s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187870982 (311h31m51.919138612s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187878215 (311h31m51.919145845s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188266736 (311h31m51.919534356s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188291242 (311h31m51.919558873s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188643465 (311h31m51.919911095s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188658473 (311h31m51.919926103s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188993774 (311h31m51.920261394s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189017548 (311h31m51.920285178s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189365152 (311h31m51.920632782s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189952657 (311h31m51.921220287s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.190704210 (311h31m51.92197184s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.191563527 (311h31m51.922831157s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.192590128 (311h31m51.923857758s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.292199435 (311h31m52.023467055s) [upload=323881]: Next returned nil segment
Aug 21 16:58:11.292208382 (311h31m52.023476002s) [upload=323881]: Waiting for error group managing segments...
Aug 21 16:58:11.305981536 (311h31m52.037249156s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.317455457 (311h31m52.048723076s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.319650365 (311h31m52.050917985s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.320890678 (311h31m52.052158297s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.325965804 (311h31m52.057233424s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.327860037 (311h31m52.059127667s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.328132208 (311h31m52.059399828s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.330010401 (311h31m52.061278021s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.335413914 (311h31m52.066681533s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.341151935 (311h31m52.072419546s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.348847890 (311h31m52.08011552s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.360184613 (311h31m52.091452233s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.366983571 (311h31m52.098251201s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392521653 (311h31m52.123789274s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392540769 (311h31m52.123808399s) [seg_pos={0 0}, upload=323881]: Segment reached optimal threshold of 14 pieces.
Aug 21 16:58:11.392827879 (311h31m52.124095509s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XaE29W27ym2RmuGdBvHeKcQZZEUbask1zsm9Bz8xLokZNGfcj): protocol: expected piece hash; context canceled; EOF
Aug 21 16:58:11.393068121 (311h31m52.124335742s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1h1gxgVjoHUkjKXA5xhTowGKi9q6wZGyzg8eqKgMHqsxF1ut5P): protocol: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection
Aug 21 16:58:11.393129707 (311h31m52.124397326s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:61860->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 21 16:58:11.393280721 (311h31m52.124548371s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.393342206 (311h31m52.124609846s) [seg_pos={0 0}, upload=323881]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 21 16:58:11.393461841 (311h31m52.124729492s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 21 16:58:11.393467823 (311h31m52.124735463s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 16:58:11.446017719 (311h31m52.177285349s) [upload=323881]: Done uploading segments...
Aug 21 16:58:11.446031615 (311h31m52.177299245s) [upload=323881]: Upload finished. err: <nil>
Duration: 1.908217692s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
321.298965ms
FAIL (2025-08-21 17:30:17 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 321.326978ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
3 out of 15 is failed (threshold=6) [..........XXX..]
duration:
75.624204ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:28:40 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-21 17:30:10 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 75.649112ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
2 out of 15 is failed (threshold=6) [......X......X.]
duration:
331.904364ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:29:19 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:30:00 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 331.932116ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..X................X........................................]
duration:
561.131443ms
FAIL (2025-08-21 17:29:29 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: invalid range 0:4194304 (size:4194304)
======LOGS======
Aug 21 16:58:11.097765980 (311h31m51.82903362s) [upload=323881]: Starting upload
Aug 21 16:58:11.097816425 (311h31m51.829084066s) [upload=323881]: Done starting upload
Aug 21 16:58:11.097974212 (311h31m51.829241862s) [upload=323881]: Uploading segments...
Aug 21 16:58:11.098124976 (311h31m51.829392606s) [upload=323881]: Got next segment. Inline: false
Aug 21 16:58:11.098190559 (311h31m51.829458199s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 16:58:11.098198244 (311h31m51.829465904s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 21 16:58:11.186959668 (311h31m51.918227288s) [seg_pos={0 0}, upload=323881]: Begin upload segment...
Aug 21 16:58:11.187010914 (311h31m51.918278574s) [seg_pos={0 0}, upload=323881]: Done begin upload segment.
Aug 21 16:58:11.187041721 (311h31m51.918309351s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187059065 (311h31m51.918326695s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187096966 (311h31m51.918364587s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187499954 (311h31m51.918767574s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187516114 (311h31m51.918783745s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187870982 (311h31m51.919138612s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.187878215 (311h31m51.919145845s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188266736 (311h31m51.919534356s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188291242 (311h31m51.919558873s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188643465 (311h31m51.919911095s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188658473 (311h31m51.919926103s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.188993774 (311h31m51.920261394s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189017548 (311h31m51.920285178s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189365152 (311h31m51.920632782s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.189952657 (311h31m51.921220287s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.190704210 (311h31m51.92197184s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.191563527 (311h31m51.922831157s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.192590128 (311h31m51.923857758s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Uploading piece...
Aug 21 16:58:11.292199435 (311h31m52.023467055s) [upload=323881]: Next returned nil segment
Aug 21 16:58:11.292208382 (311h31m52.023476002s) [upload=323881]: Waiting for error group managing segments...
Aug 21 16:58:11.305981536 (311h31m52.037249156s) [noise=true , address=104.36.82.11:4118    , piece_id=MFWA4J22CPFE4HEYFXZISYJDAKWCF3HEYHVCYKSFQY5N27QT2O6A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.317455457 (311h31m52.048723076s) [noise=true , address=77.90.31.11:4582     , piece_id=7I4O5F46N3EAAFKUSTBOA3DY5NF7PQHNILWM5GFZ5CEIGRZANDRQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.319650365 (311h31m52.050917985s) [noise=true , address=77.90.31.18:4812     , piece_id=GRFZT75OBWQBGFZ3I3QYOGWHIFXJCZHKOGQOG34ZW6QG7BNBNGBQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.320890678 (311h31m52.052158297s) [noise=true , address=104.36.82.49:4012    , piece_id=CQSJDRQSYII2TZ4IXSILK3IN4WMGVOJEHPQJBPWPYII5HQ7G2B3A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.325965804 (311h31m52.057233424s) [noise=true , address=77.90.31.15:4719     , piece_id=SQOIUIMMOLGQHURPXKR4E3SYUKDJDJNZ75Q4RXA3W2E4JOG5WXLQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.327860037 (311h31m52.059127667s) [noise=true , address=45.91.116.11:4142    , piece_id=FTHDX3KRMQV2VPJA42JPTD5LDPAQCC6C2D6ODRZ7M4ZASZDJCSYA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.328132208 (311h31m52.059399828s) [noise=true , address=45.91.116.5:4312     , piece_id=QCY54KDIJ3SFLIE2X6KBQD2UIEWMGYBOL33JKFLWGXZGRTB42YQQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.330010401 (311h31m52.061278021s) [noise=true , address=77.90.31.10:4548     , piece_id=3IWDAKTWMTSIYUHS4YSHUGNPZXYIOTCE523RYZCYL2FEW226R45A, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.335413914 (311h31m52.066681533s) [noise=true , address=77.90.31.13:4651     , piece_id=KVYWHATVMKX3WZB7UWCIPQ5TIXHLAO7TWECK4XRQDILQ4HOUSB3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.341151935 (311h31m52.072419546s) [noise=true , address=104.36.82.9:4022     , piece_id=XHQFV3SHMAT7WOOWG6U5QRRO3VBLZUE2SV7N5N4IDWWB6MPAT7JA, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.348847890 (311h31m52.08011552s) [noise=true , address=104.36.82.6:4907     , piece_id=ZVU2TK7PXTZBHLLU657Q5U5BS33QIIGR5WBBUE23LMRWNYSGIJEQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.360184613 (311h31m52.091452233s) [noise=true , address=104.36.82.13:4209    , piece_id=AJ5OPQIYSANI4EFJQYTCSC4OYA2KV4PR23YNLROCLL62ZNT2CINQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.366983571 (311h31m52.098251201s) [noise=true , address=77.90.31.19:4851     , piece_id=YZ7SGALUQPTAHO5ZARMCVMADX5AGH6BQNCN5UTC74JPLPEWALEDQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392521653 (311h31m52.123789274s) [noise=true , address=104.36.82.45:4917    , piece_id=HHWHHUVVM76NSJNSFRSV5L2TJM7WABLPY3UMVM7CEZM44NKMXRHQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.392540769 (311h31m52.123808399s) [seg_pos={0 0}, upload=323881]: Segment reached optimal threshold of 14 pieces.
Aug 21 16:58:11.392827879 (311h31m52.124095509s) [noise=true , address=104.36.82.14:4236    , piece_id=EOTHS7XHUD5YNTOQOVTFR6EIRGREL4VBL2KLS5LJVQJKTMRWWUVQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XaE29W27ym2RmuGdBvHeKcQZZEUbask1zsm9Bz8xLokZNGfcj): protocol: expected piece hash; context canceled; EOF
Aug 21 16:58:11.393068121 (311h31m52.124335742s) [noise=true , address=45.91.116.11:4141    , piece_id=6OUW5WSYNG5GABS6BH3NNP3WRPYAK3SOVJDSXLMPICFMSNWD3O3Q, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1h1gxgVjoHUkjKXA5xhTowGKi9q6wZGyzg8eqKgMHqsxF1ut5P): protocol: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:47570->45.91.116.11:4141: use of closed network connection
Aug 21 16:58:11.393129707 (311h31m52.124397326s) [noise=true , address=79.127.205.234:4616  , piece_id=D3SSTIPP6BS4NL5IS3JTR6JTHG32VWVD6HIU27WB2HXRTI7SL4TQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PuFfyFT8z7TwZZsBhMRPx8HtLBqELmEzxbbsuG9Ljg9KKz3rm): protocol: read tcp 5.161.89.252:61860->79.127.205.234:4616: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 21 16:58:11.393280721 (311h31m52.124548371s) [noise=true , address=104.36.82.7:4920     , piece_id=7LGSALS6MO7SEPNO5KGHKACZ6CSAMJMV2GYHYY6BBVGFOXHRLPSQ, seg_pos={0 0}, upload=323881]: Done uploading piece. err: <nil>
Aug 21 16:58:11.393342206 (311h31m52.124609846s) [seg_pos={0 0}, upload=323881]: Done waiting for segment. successful: 15 optimal: 14 errs: <nil>
Aug 21 16:58:11.393461841 (311h31m52.124729492s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 21 16:58:11.393467823 (311h31m52.124735463s) [upload=323881]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 16:58:11.446017719 (311h31m52.177285349s) [upload=323881]: Done uploading segments...
Aug 21 16:58:11.446031615 (311h31m52.177299245s) [upload=323881]: Upload finished. err: <nil>
Duration: 561.156521ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
2 out of 15 is failed (threshold=6) [.......X......X]
duration:
1.01545296s
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Thu, 21 Aug 2025 17:29:19 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-21 17:29:19 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.015470043s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
207.136397ms
FAIL (2025-08-21 17:29:02 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 21 17:25:11.548962484 (311h58m52.280230114s) [upload=324350]: Starting upload
Aug 21 17:25:11.549002850 (311h58m52.28027047s) [upload=324350]: Done starting upload
Aug 21 17:25:11.549054988 (311h58m52.280322608s) [upload=324350]: Uploading segments...
Aug 21 17:25:11.549092488 (311h58m52.280360118s) [upload=324350]: Got next segment. Inline: true
Aug 21 17:25:11.549096727 (311h58m52.280364357s) [upload=324350]: Waiting for error group managing segments...
Aug 21 17:25:11.549145848 (311h58m52.280413478s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 21 17:25:11.549150006 (311h58m52.280417626s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 21 17:25:11.549152240 (311h58m52.280419871s) [upload=324350]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 21 17:25:11.553018232 (311h58m52.284285862s) [upload=324350]: Done uploading segments...
Aug 21 17:25:11.553025245 (311h58m52.284292866s) [upload=324350]: Upload finished. err: metaclient: Too Many Requests
Duration: 207.180931ms core uplink us-select-1 us1-placement-20 metainfo-projects