Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 117413 reports (Page 1597 of 2349)
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
3 out of 15 is failed (threshold=6) [X.......XX.....]
duration:
78.503039ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:35 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:36:38 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 78.520742ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
2 out of 15 is failed (threshold=6) [.X........X....]
duration:
58.141258ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:58 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:36:35 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 58.166946ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [X.........X......X..........X...............................]
duration:
385.098488ms
FAIL (2025-08-20 16:36:14 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 385.118014ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
3 out of 15 is failed (threshold=6) [.X.......XX....]
duration:
106.401917ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:35 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:35:56 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 106.428847ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
2 out of 15 is failed (threshold=6) [..X........X...]
duration:
199.426873ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:58 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:35:54 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 199.447713ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [................................................X....X......]
duration:
515.73642ms
FAIL (2025-08-20 16:35:53 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:32:05.324046131 (287h5m46.055313751s) [upload=298561]: Starting upload
Aug 20 16:32:05.324087819 (287h5m46.055355449s) [upload=298561]: Done starting upload
Aug 20 16:32:05.324385969 (287h5m46.0556536s) [upload=298561]: Uploading segments...
Aug 20 16:32:05.324417028 (287h5m46.055684658s) [upload=298561]: Got next segment. Inline: false
Aug 20 16:32:05.324441945 (287h5m46.055709564s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:32:05.324445723 (287h5m46.055713343s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:32:05.327260850 (287h5m46.05852848s) [upload=298561]: Next segment err: metaclient: Too Many Requests
Aug 20 16:32:05.327271079 (287h5m46.058538709s) [upload=298561]: Done uploading segments...
Aug 20 16:32:05.327279075 (287h5m46.058546705s) [upload=298561]: Upload finished. err: metaclient: Too Many Requests
Duration: 515.806513ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [.X.........X......X..........X..............................]
duration:
467.707796ms
FAIL (2025-08-20 16:35:12 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 468.082419ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
3 out of 15 is failed (threshold=6) [..X.......XX...]
duration:
73.575712ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:35 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:35:09 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 73.614574ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
2 out of 15 is failed (threshold=6) [...X........X..]
duration:
90.612605ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:58 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:34:56 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 90.629557ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [..X.........X......X..........X.............................]
duration:
524.052412ms
FAIL (2025-08-20 16:34:53 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 524.082979ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.................................................X....X.....]
duration:
2.14006379s
FAIL (2025-08-20 16:34:44 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:32:05.324046131 (287h5m46.055313751s) [upload=298561]: Starting upload
Aug 20 16:32:05.324087819 (287h5m46.055355449s) [upload=298561]: Done starting upload
Aug 20 16:32:05.324385969 (287h5m46.0556536s) [upload=298561]: Uploading segments...
Aug 20 16:32:05.324417028 (287h5m46.055684658s) [upload=298561]: Got next segment. Inline: false
Aug 20 16:32:05.324441945 (287h5m46.055709564s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:32:05.324445723 (287h5m46.055713343s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:32:05.327260850 (287h5m46.05852848s) [upload=298561]: Next segment err: metaclient: Too Many Requests
Aug 20 16:32:05.327271079 (287h5m46.058538709s) [upload=298561]: Done uploading segments...
Aug 20 16:32:05.327279075 (287h5m46.058546705s) [upload=298561]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.14008979s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..................................................X....X....]
duration:
1.063112496s
FAIL (2025-08-20 16:34:19 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:32:05.324046131 (287h5m46.055313751s) [upload=298561]: Starting upload
Aug 20 16:32:05.324087819 (287h5m46.055355449s) [upload=298561]: Done starting upload
Aug 20 16:32:05.324385969 (287h5m46.0556536s) [upload=298561]: Uploading segments...
Aug 20 16:32:05.324417028 (287h5m46.055684658s) [upload=298561]: Got next segment. Inline: false
Aug 20 16:32:05.324441945 (287h5m46.055709564s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:32:05.324445723 (287h5m46.055713343s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:32:05.327260850 (287h5m46.05852848s) [upload=298561]: Next segment err: metaclient: Too Many Requests
Aug 20 16:32:05.327271079 (287h5m46.058538709s) [upload=298561]: Done uploading segments...
Aug 20 16:32:05.327279075 (287h5m46.058546705s) [upload=298561]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.063141762s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [...X.........X......X..........X............................]
duration:
558.752415ms
FAIL (2025-08-20 16:34:16 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 558.776459ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
3 out of 15 is failed (threshold=6) [...X.......XX..]
duration:
79.132001ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:35 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:34:12 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 79.150767ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
2 out of 15 is failed (threshold=6) [....X........X.]
duration:
193.846902ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:58 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:34:11 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 193.868302ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [...................................................X....X...]
duration:
1.057876823s
FAIL (2025-08-20 16:33:49 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:32:05.324046131 (287h5m46.055313751s) [upload=298561]: Starting upload
Aug 20 16:32:05.324087819 (287h5m46.055355449s) [upload=298561]: Done starting upload
Aug 20 16:32:05.324385969 (287h5m46.0556536s) [upload=298561]: Uploading segments...
Aug 20 16:32:05.324417028 (287h5m46.055684658s) [upload=298561]: Got next segment. Inline: false
Aug 20 16:32:05.324441945 (287h5m46.055709564s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:32:05.324445723 (287h5m46.055713343s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:32:05.327260850 (287h5m46.05852848s) [upload=298561]: Next segment err: metaclient: Too Many Requests
Aug 20 16:32:05.327271079 (287h5m46.058538709s) [upload=298561]: Done uploading segments...
Aug 20 16:32:05.327279075 (287h5m46.058546705s) [upload=298561]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.057896179s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
3 out of 15 is failed (threshold=6) [....X.......XX.]
duration:
59.63335ms
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:35 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:33:44 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 59.654039ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [....X.........X......X..........X...........................]
duration:
284.971618ms
FAIL (2025-08-20 16:33:29 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 284.996234ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [....................................................X....X..]
duration:
1.176335595s
FAIL (2025-08-20 16:33:01 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:32:05.324046131 (287h5m46.055313751s) [upload=298561]: Starting upload
Aug 20 16:32:05.324087819 (287h5m46.055355449s) [upload=298561]: Done starting upload
Aug 20 16:32:05.324385969 (287h5m46.0556536s) [upload=298561]: Uploading segments...
Aug 20 16:32:05.324417028 (287h5m46.055684658s) [upload=298561]: Got next segment. Inline: false
Aug 20 16:32:05.324441945 (287h5m46.055709564s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:32:05.324445723 (287h5m46.055713343s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:32:05.327260850 (287h5m46.05852848s) [upload=298561]: Next segment err: metaclient: Too Many Requests
Aug 20 16:32:05.327271079 (287h5m46.058538709s) [upload=298561]: Done uploading segments...
Aug 20 16:32:05.327279075 (287h5m46.058546705s) [upload=298561]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.176356413s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
2 out of 15 is failed (threshold=6) [.....X........X]
duration:
1.083460722s
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:58 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:32:58 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.083473807s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [.....X.........X......X..........X..........................]
duration:
271.689177ms
FAIL (2025-08-20 16:32:55 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 271.710567ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
3 out of 15 is failed (threshold=6) [.....X.......XX]
duration:
1.009491061s
remote:
"tcp:18.211.166.153:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:32:35 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:32:35 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.009512241s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.....................................................X....X.]
duration:
669.990638ms
FAIL (2025-08-20 16:32:23 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:32:05.324046131 (287h5m46.055313751s) [upload=298561]: Starting upload
Aug 20 16:32:05.324087819 (287h5m46.055355449s) [upload=298561]: Done starting upload
Aug 20 16:32:05.324385969 (287h5m46.0556536s) [upload=298561]: Uploading segments...
Aug 20 16:32:05.324417028 (287h5m46.055684658s) [upload=298561]: Got next segment. Inline: false
Aug 20 16:32:05.324441945 (287h5m46.055709564s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:32:05.324445723 (287h5m46.055713343s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:32:05.327260850 (287h5m46.05852848s) [upload=298561]: Next segment err: metaclient: Too Many Requests
Aug 20 16:32:05.327271079 (287h5m46.058538709s) [upload=298561]: Done uploading segments...
Aug 20 16:32:05.327279075 (287h5m46.058546705s) [upload=298561]: Upload finished. err: metaclient: Too Many Requests
Duration: 670.015624ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
2 out of 15 is failed (threshold=6) [......X.......X]
duration:
1.033353134s
remote:
"tcp:54.243.86.28:443"
response headers:
Date=Wed, 20 Aug 2025 16:32:08 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:32:08 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 1.033376898s core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
2 out of 60 is failed (threshold=3) [......................................................X....X]
duration:
20.401995ms
FAIL (2025-08-20 16:32:05 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:32:05.324046131 (287h5m46.055313751s) [upload=298561]: Starting upload
Aug 20 16:32:05.324087819 (287h5m46.055355449s) [upload=298561]: Done starting upload
Aug 20 16:32:05.324385969 (287h5m46.0556536s) [upload=298561]: Uploading segments...
Aug 20 16:32:05.324417028 (287h5m46.055684658s) [upload=298561]: Got next segment. Inline: false
Aug 20 16:32:05.324441945 (287h5m46.055709564s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:32:05.324445723 (287h5m46.055713343s) [upload=298561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:32:05.327260850 (287h5m46.05852848s) [upload=298561]: Next segment err: metaclient: Too Many Requests
Aug 20 16:32:05.327271079 (287h5m46.058538709s) [upload=298561]: Done uploading segments...
Aug 20 16:32:05.327279075 (287h5m46.058546705s) [upload=298561]: Upload finished. err: metaclient: Too Many Requests
Duration: 20.41547ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [......X.........X......X..........X.........................]
duration:
185.831862ms
FAIL (2025-08-20 16:31:59 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 185.85753ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
1 out of 15 is failed (threshold=6) [......X........]
duration:
121.535022ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:26:51 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:31:49 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 121.559419ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
1 out of 15 is failed (threshold=6) [.......X.......]
duration:
114.635915ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Wed, 20 Aug 2025 16:26:36 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:31:13 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 114.659949ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
1 out of 15 is failed (threshold=6) [.......X.......]
duration:
82.04432ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:26:51 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:30:56 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 82.066201ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
2.131102447s
FAIL (2025-08-20 16:30:52 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:27:58.816567678 (287h1m39.547835308s) [upload=298493]: Starting upload
Aug 20 16:27:58.816638121 (287h1m39.547905751s) [upload=298493]: Done starting upload
Aug 20 16:27:58.816764167 (287h1m39.548031797s) [upload=298493]: Uploading segments...
Aug 20 16:27:58.816997495 (287h1m39.548265126s) [upload=298493]: Got next segment. Inline: false
Aug 20 16:27:58.817052108 (287h1m39.548319759s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:27:58.817057719 (287h1m39.548325369s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:27:58.820016617 (287h1m39.551284248s) [upload=298493]: Next segment err: metaclient: Too Many Requests
Aug 20 16:27:58.820031004 (287h1m39.551298644s) [upload=298493]: Done uploading segments...
Aug 20 16:27:58.820044920 (287h1m39.55131256s) [upload=298493]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.13111975s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [.......X.........X......X..........X........................]
duration:
589.639959ms
FAIL (2025-08-20 16:30:47 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 589.658615ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
1 out of 15 is failed (threshold=6) [........X......]
duration:
67.28104ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Wed, 20 Aug 2025 16:26:36 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:30:39 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 67.301598ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
1 out of 15 is failed (threshold=6) [........X......]
duration:
69.111467ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:26:51 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:29:56 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 69.131013ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
927.163548ms
FAIL (2025-08-20 16:29:47 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:27:58.816567678 (287h1m39.547835308s) [upload=298493]: Starting upload
Aug 20 16:27:58.816638121 (287h1m39.547905751s) [upload=298493]: Done starting upload
Aug 20 16:27:58.816764167 (287h1m39.548031797s) [upload=298493]: Uploading segments...
Aug 20 16:27:58.816997495 (287h1m39.548265126s) [upload=298493]: Got next segment. Inline: false
Aug 20 16:27:58.817052108 (287h1m39.548319759s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:27:58.817057719 (287h1m39.548325369s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:27:58.820016617 (287h1m39.551284248s) [upload=298493]: Next segment err: metaclient: Too Many Requests
Aug 20 16:27:58.820031004 (287h1m39.551298644s) [upload=298493]: Done uploading segments...
Aug 20 16:27:58.820044920 (287h1m39.55131256s) [upload=298493]: Upload finished. err: metaclient: Too Many Requests
Duration: 927.179547ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [........X.........X......X..........X.......................]
duration:
451.871511ms
FAIL (2025-08-20 16:29:46 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 451.894855ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
1 out of 15 is failed (threshold=6) [.........X.....]
duration:
60.84025ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Wed, 20 Aug 2025 16:26:36 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:29:37 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 60.854296ms core website
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
1 out of 15 is failed (threshold=6) [.........X.....]
duration:
91.73889ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:26:51 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:29:32 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 91.758226ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
1 out of 15 is failed (threshold=6) [..........X....]
duration:
73.388881ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Wed, 20 Aug 2025 16:26:36 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:29:22 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 73.411543ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
1.533206418s
FAIL (2025-08-20 16:29:19 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:27:58.816567678 (287h1m39.547835308s) [upload=298493]: Starting upload
Aug 20 16:27:58.816638121 (287h1m39.547905751s) [upload=298493]: Done starting upload
Aug 20 16:27:58.816764167 (287h1m39.548031797s) [upload=298493]: Uploading segments...
Aug 20 16:27:58.816997495 (287h1m39.548265126s) [upload=298493]: Got next segment. Inline: false
Aug 20 16:27:58.817052108 (287h1m39.548319759s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:27:58.817057719 (287h1m39.548325369s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:27:58.820016617 (287h1m39.551284248s) [upload=298493]: Next segment err: metaclient: Too Many Requests
Aug 20 16:27:58.820031004 (287h1m39.551298644s) [upload=298493]: Done uploading segments...
Aug 20 16:27:58.820044920 (287h1m39.55131256s) [upload=298493]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.533235694s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [.........X.........X......X..........X......................]
duration:
1.470479456s
FAIL (2025-08-20 16:28:59 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.470495687s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
1 out of 15 is failed (threshold=6) [..........X....]
duration:
95.367989ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:26:51 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:28:54 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 95.384861ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
1 out of 15 is failed (threshold=6) [...........X...]
duration:
57.968664ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Wed, 20 Aug 2025 16:26:36 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:28:52 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 57.990565ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
1.849732363s
FAIL (2025-08-20 16:28:24 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:27:58.816567678 (287h1m39.547835308s) [upload=298493]: Starting upload
Aug 20 16:27:58.816638121 (287h1m39.547905751s) [upload=298493]: Done starting upload
Aug 20 16:27:58.816764167 (287h1m39.548031797s) [upload=298493]: Uploading segments...
Aug 20 16:27:58.816997495 (287h1m39.548265126s) [upload=298493]: Got next segment. Inline: false
Aug 20 16:27:58.817052108 (287h1m39.548319759s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:27:58.817057719 (287h1m39.548325369s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:27:58.820016617 (287h1m39.551284248s) [upload=298493]: Next segment err: metaclient: Too Many Requests
Aug 20 16:27:58.820031004 (287h1m39.551298644s) [upload=298493]: Done uploading segments...
Aug 20 16:27:58.820044920 (287h1m39.55131256s) [upload=298493]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.849754586s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
1 out of 15 is failed (threshold=6) [...........X...]
duration:
69.088886ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:26:51 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:28:21 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 69.11281ms core website
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect)
antiblip:
1 out of 15 is failed (threshold=6) [............X..]
duration:
63.776807ms
remote:
"tcp:18.211.166.153:443"
response headers:
Date=Wed, 20 Aug 2025 16:26:36 GMT
Content-Type=text/html
Content-Length=164
X-Cluster-Name=us-east-1-prod-hosting-red
target:
https://www.storj.io/dcs-satellites
FAIL (2025-08-20 16:28:21 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 63.807314ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [..........X.........X......X..........X.....................]
duration:
160.847352ms
FAIL (2025-08-20 16:28:14 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 160.873161ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
1 out of 15 is failed (threshold=6) [............X..]
duration:
113.207891ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:26:51 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:28:05 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 113.225153ms core website
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
20.695727ms
FAIL (2025-08-20 16:27:58 UTC)
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:27:58.816567678 (287h1m39.547835308s) [upload=298493]: Starting upload
Aug 20 16:27:58.816638121 (287h1m39.547905751s) [upload=298493]: Done starting upload
Aug 20 16:27:58.816764167 (287h1m39.548031797s) [upload=298493]: Uploading segments...
Aug 20 16:27:58.816997495 (287h1m39.548265126s) [upload=298493]: Got next segment. Inline: false
Aug 20 16:27:58.817052108 (287h1m39.548319759s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:27:58.817057719 (287h1m39.548325369s) [upload=298493]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 20 16:27:58.820016617 (287h1m39.551284248s) [upload=298493]: Next segment err: metaclient: Too Many Requests
Aug 20 16:27:58.820031004 (287h1m39.551298644s) [upload=298493]: Done uploading segments...
Aug 20 16:27:58.820044920 (287h1m39.55131256s) [upload=298493]: Upload finished. err: metaclient: Too Many Requests
Duration: 20.712378ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)
antiblip:
4 out of 60 is failed (threshold=3) [...........X.........X......X..........X....................]
duration:
208.443021ms
FAIL (2025-08-20 16:27:35 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: too many requests
======LOGS======
Aug 20 16:12:54.720936118 (286h46m35.452203779s) [upload=298240]: Starting upload
Aug 20 16:12:54.721001581 (286h46m35.452269231s) [upload=298240]: Done starting upload
Aug 20 16:12:54.721062465 (286h46m35.452330126s) [upload=298240]: Uploading segments...
Aug 20 16:12:54.721102501 (286h46m35.452370151s) [upload=298240]: Got next segment. Inline: true
Aug 20 16:12:54.721107880 (286h46m35.452375531s) [upload=298240]: Waiting for error group managing segments...
Aug 20 16:12:54.721153737 (286h46m35.452421387s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 20 16:12:54.721159748 (286h46m35.452427398s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline
Aug 20 16:12:54.721163615 (286h46m35.452431276s) [upload=298240]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 20 16:12:54.723615261 (286h46m35.454882912s) [upload=298240]: Done uploading segments...
Aug 20 16:12:54.723626853 (286h46m35.454894503s) [upload=298240]: Upload finished. err: metaclient: Too Many Requests
Duration: 208.475773ms core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.HTTP{"GET" "https://www.storj.io/"} failure (antiblip window 6/15): (http.www_storj_io)
antiblip:
1 out of 15 is failed (threshold=6) [.............X.]
duration:
341.431124ms
remote:
"tcp:54.243.86.28:443"
response headers:
X-Cluster-Name=us-east-1-prod-hosting-red
Date=Wed, 20 Aug 2025 16:26:51 GMT
Content-Type=text/html
Content-Length=164
target:
https://www.storj.io/
FAIL (2025-08-20 16:27:32 UTC)
orbiter warning: unexpected status code "504 Gateway Time-out"
"<html>\r\n<head><title>504 Gateway Time-out</title></head>\r\n<body>\r\n<center><h1>504 Gateway Time-out</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n"
Duration: 341.453947ms core website