Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................X........X........................] duration:
3.969492399s
FAIL (2025-08-16 14:39:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 3.969555668s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
473.316752ms target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt
FAIL (2025-08-16 14:39:49 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 136.0.77.2:443: i/o timeout
Duration: 473.378648ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
30.000462876s target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt
FAIL (2025-08-16 14:39:45 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": dial tcp 136.0.77.2:443: i/o timeout
Duration: 30.000483655s
edge
linksharing
bgp
ap1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
20.657177463s
FAIL (2025-08-16 14:39:43 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 136.0.77.2:443: i/o timeout
Duration: 20.657193154s
edge
authservice
http
bgp
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
25.577416943s
FAIL (2025-08-16 14:39:38 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": net/http: TLS handshake timeout
Duration: 25.577438384s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
812.293237ms target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-16 14:39:33 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 136.0.77.2:443: i/o timeout
Duration: 812.312422ms
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............................X......................X.......] duration:
174.610936ms
FAIL (2025-08-16 14:39:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:35:30.464493409 (189h9m11.195761039s) [upload=196828]: Starting upload Aug 16 14:35:30.464534296 (189h9m11.195801926s) [upload=196828]: Done starting upload Aug 16 14:35:30.464585943 (189h9m11.195853563s) [upload=196828]: Uploading segments... Aug 16 14:35:30.464611672 (189h9m11.195879292s) [upload=196828]: Got next segment. Inline: true Aug 16 14:35:30.464615298 (189h9m11.195882928s) [upload=196828]: Waiting for error group managing segments... Aug 16 14:35:30.464648020 (189h9m11.19591565s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:35:30.464651457 (189h9m11.195919087s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:35:30.464653831 (189h9m11.195921451s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:35:30.515094838 (189h9m11.246362468s) [upload=196828]: Done uploading segments... Aug 16 14:35:30.515109124 (189h9m11.246376754s) [upload=196828]: Upload finished. err: metaclient: Too Many Requests
Duration: 174.628458ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777"} failure (antiblip window 3/60): (authservice.eu1)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001296906s
FAIL (2025-08-16 14:39:22 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 136.0.77.2:443: i/o timeout
Duration: 30.001336762s
edge
authservice
http
bgp
eu1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777"} failure (antiblip window 3/60): (authservice.ap1)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.0005345s
FAIL (2025-08-16 14:39:18 UTC)
orbiter warning: Post "https://auth.storjshare.io/v1/access": dial tcp 136.0.77.2:443: i/o timeout
Duration: 30.00055538s
edge
authservice
http
bgp
ap1
|
checkers.HTTP{"GET" "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222", ExpectedContent: "Hello, world!"} failure (antiblip window 3/60): (http.us1.gateway)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.000488554s target:
https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222
FAIL (2025-08-16 14:39:17 UTC)
orbiter warning: Get "https://uptime-checks.gateway.storjshare.io/hello-world.txt?AWSAccessKeyId=jw7f74fzbtwblz3mwqolmnivikxq&Signature=azz0x2J96uUqIe6sG77G4wqaf50%3D&Expires=1771549222": dial tcp 136.0.77.2:443: i/o timeout
Duration: 30.000535943s
edge
gateway
vhost
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................X........X.......................] duration:
760.85302ms
FAIL (2025-08-16 14:39:07 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 760.87959ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.821877213s
FAIL (2025-08-16 14:39:06 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.821895157s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............................X......................X......] duration:
249.605239ms
FAIL (2025-08-16 14:39:04 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:35:30.464493409 (189h9m11.195761039s) [upload=196828]: Starting upload Aug 16 14:35:30.464534296 (189h9m11.195801926s) [upload=196828]: Done starting upload Aug 16 14:35:30.464585943 (189h9m11.195853563s) [upload=196828]: Uploading segments... Aug 16 14:35:30.464611672 (189h9m11.195879292s) [upload=196828]: Got next segment. Inline: true Aug 16 14:35:30.464615298 (189h9m11.195882928s) [upload=196828]: Waiting for error group managing segments... Aug 16 14:35:30.464648020 (189h9m11.19591565s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:35:30.464651457 (189h9m11.195919087s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:35:30.464653831 (189h9m11.195921451s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:35:30.515094838 (189h9m11.246362468s) [upload=196828]: Done uploading segments... Aug 16 14:35:30.515109124 (189h9m11.246376754s) [upload=196828]: Upload finished. err: metaclient: Too Many Requests
Duration: 249.632849ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............................X......................X.....] duration:
312.795098ms
FAIL (2025-08-16 14:38:31 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:35:30.464493409 (189h9m11.195761039s) [upload=196828]: Starting upload Aug 16 14:35:30.464534296 (189h9m11.195801926s) [upload=196828]: Done starting upload Aug 16 14:35:30.464585943 (189h9m11.195853563s) [upload=196828]: Uploading segments... Aug 16 14:35:30.464611672 (189h9m11.195879292s) [upload=196828]: Got next segment. Inline: true Aug 16 14:35:30.464615298 (189h9m11.195882928s) [upload=196828]: Waiting for error group managing segments... Aug 16 14:35:30.464648020 (189h9m11.19591565s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:35:30.464651457 (189h9m11.195919087s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:35:30.464653831 (189h9m11.195921451s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:35:30.515094838 (189h9m11.246362468s) [upload=196828]: Done uploading segments... Aug 16 14:35:30.515109124 (189h9m11.246376754s) [upload=196828]: Upload finished. err: metaclient: Too Many Requests
Duration: 312.842867ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................X........X......................] duration:
3.489694564s
FAIL (2025-08-16 14:38:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 3.489716426s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.343329376s
FAIL (2025-08-16 14:38:07 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 6.343354133s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................................X......................X....] duration:
5.24170857s
FAIL (2025-08-16 14:37:50 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:35:30.464493409 (189h9m11.195761039s) [upload=196828]: Starting upload Aug 16 14:35:30.464534296 (189h9m11.195801926s) [upload=196828]: Done starting upload Aug 16 14:35:30.464585943 (189h9m11.195853563s) [upload=196828]: Uploading segments... Aug 16 14:35:30.464611672 (189h9m11.195879292s) [upload=196828]: Got next segment. Inline: true Aug 16 14:35:30.464615298 (189h9m11.195882928s) [upload=196828]: Waiting for error group managing segments... Aug 16 14:35:30.464648020 (189h9m11.19591565s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:35:30.464651457 (189h9m11.195919087s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:35:30.464653831 (189h9m11.195921451s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:35:30.515094838 (189h9m11.246362468s) [upload=196828]: Done uploading segments... Aug 16 14:35:30.515109124 (189h9m11.246376754s) [upload=196828]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.241775867s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
6.201913845s
FAIL (2025-08-16 14:37:30 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 6.201941277s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................X........X.....................] duration:
6.2835357s
FAIL (2025-08-16 14:37:29 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.283565567s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................................X......................X...] duration:
6.690769474s
FAIL (2025-08-16 14:37:20 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:35:30.464493409 (189h9m11.195761039s) [upload=196828]: Starting upload Aug 16 14:35:30.464534296 (189h9m11.195801926s) [upload=196828]: Done starting upload Aug 16 14:35:30.464585943 (189h9m11.195853563s) [upload=196828]: Uploading segments... Aug 16 14:35:30.464611672 (189h9m11.195879292s) [upload=196828]: Got next segment. Inline: true Aug 16 14:35:30.464615298 (189h9m11.195882928s) [upload=196828]: Waiting for error group managing segments... Aug 16 14:35:30.464648020 (189h9m11.19591565s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:35:30.464651457 (189h9m11.195919087s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:35:30.464653831 (189h9m11.195921451s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:35:30.515094838 (189h9m11.246362468s) [upload=196828]: Done uploading segments... Aug 16 14:35:30.515109124 (189h9m11.246376754s) [upload=196828]: Upload finished. err: metaclient: Too Many Requests
Duration: 6.690788701s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.112502882s
FAIL (2025-08-16 14:36:52 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.112525294s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................X........X....................] duration:
4.923582406s
FAIL (2025-08-16 14:36:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 4.923606342s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................................X......................X..] duration:
5.025816172s
FAIL (2025-08-16 14:36:41 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:35:30.464493409 (189h9m11.195761039s) [upload=196828]: Starting upload Aug 16 14:35:30.464534296 (189h9m11.195801926s) [upload=196828]: Done starting upload Aug 16 14:35:30.464585943 (189h9m11.195853563s) [upload=196828]: Uploading segments... Aug 16 14:35:30.464611672 (189h9m11.195879292s) [upload=196828]: Got next segment. Inline: true Aug 16 14:35:30.464615298 (189h9m11.195882928s) [upload=196828]: Waiting for error group managing segments... Aug 16 14:35:30.464648020 (189h9m11.19591565s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:35:30.464651457 (189h9m11.195919087s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:35:30.464653831 (189h9m11.195921451s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:35:30.515094838 (189h9m11.246362468s) [upload=196828]: Done uploading segments... Aug 16 14:35:30.515109124 (189h9m11.246376754s) [upload=196828]: Upload finished. err: metaclient: Too Many Requests
Duration: 5.025839145s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...................................X......................X.] duration:
4.27500146s
FAIL (2025-08-16 14:36:22 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:35:30.464493409 (189h9m11.195761039s) [upload=196828]: Starting upload Aug 16 14:35:30.464534296 (189h9m11.195801926s) [upload=196828]: Done starting upload Aug 16 14:35:30.464585943 (189h9m11.195853563s) [upload=196828]: Uploading segments... Aug 16 14:35:30.464611672 (189h9m11.195879292s) [upload=196828]: Got next segment. Inline: true Aug 16 14:35:30.464615298 (189h9m11.195882928s) [upload=196828]: Waiting for error group managing segments... Aug 16 14:35:30.464648020 (189h9m11.19591565s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:35:30.464651457 (189h9m11.195919087s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:35:30.464653831 (189h9m11.195921451s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:35:30.515094838 (189h9m11.246362468s) [upload=196828]: Done uploading segments... Aug 16 14:35:30.515109124 (189h9m11.246376754s) [upload=196828]: Upload finished. err: metaclient: Too Many Requests
Duration: 4.275024194s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
6.258037892s
FAIL (2025-08-16 14:36:14 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 6.258060034s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................X........X...................] duration:
4.1848404s
FAIL (2025-08-16 14:36:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 4.184860567s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................X........X..................] duration:
3.231377838s
FAIL (2025-08-16 14:35:45 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 3.231406353s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [X...................................X......................X] duration:
64.019601ms
FAIL (2025-08-16 14:35:30 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:35:30.464493409 (189h9m11.195761039s) [upload=196828]: Starting upload Aug 16 14:35:30.464534296 (189h9m11.195801926s) [upload=196828]: Done starting upload Aug 16 14:35:30.464585943 (189h9m11.195853563s) [upload=196828]: Uploading segments... Aug 16 14:35:30.464611672 (189h9m11.195879292s) [upload=196828]: Got next segment. Inline: true Aug 16 14:35:30.464615298 (189h9m11.195882928s) [upload=196828]: Waiting for error group managing segments... Aug 16 14:35:30.464648020 (189h9m11.19591565s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:35:30.464651457 (189h9m11.195919087s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:35:30.464653831 (189h9m11.195921451s) [upload=196828]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:35:30.515094838 (189h9m11.246362468s) [upload=196828]: Done uploading segments... Aug 16 14:35:30.515109124 (189h9m11.246376754s) [upload=196828]: Upload finished. err: metaclient: Too Many Requests
Duration: 64.036903ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................X........X.................] duration:
3.875507516s
FAIL (2025-08-16 14:35:29 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 3.875528175s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.X...................................X......................] duration:
938.85702ms
FAIL (2025-08-16 14:35:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:19:55.392348603 (188h53m36.123616253s) [upload=196558]: Starting upload Aug 16 14:19:55.392413835 (188h53m36.123681465s) [upload=196558]: Done starting upload Aug 16 14:19:55.392478257 (188h53m36.123745877s) [upload=196558]: Uploading segments... Aug 16 14:19:55.392524073 (188h53m36.123791712s) [upload=196558]: Got next segment. Inline: true Aug 16 14:19:55.392532609 (188h53m36.123800229s) [upload=196558]: Waiting for error group managing segments... Aug 16 14:19:55.392576782 (188h53m36.123844412s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:19:55.392583505 (188h53m36.123851135s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:19:55.392588043 (188h53m36.123855673s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:19:55.415388546 (188h53m36.146656196s) [upload=196558]: Done uploading segments... Aug 16 14:19:55.415406199 (188h53m36.146673849s) [upload=196558]: Upload finished. err: metaclient: Too Many Requests
Duration: 938.88322ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................................X........X................] duration:
2.910843977s
FAIL (2025-08-16 14:35:10 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.910864627s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
7.007109131s
FAIL (2025-08-16 14:35:02 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 7.007127356s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..X...................................X.....................] duration:
293.665646ms
FAIL (2025-08-16 14:34:38 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:19:55.392348603 (188h53m36.123616253s) [upload=196558]: Starting upload Aug 16 14:19:55.392413835 (188h53m36.123681465s) [upload=196558]: Done starting upload Aug 16 14:19:55.392478257 (188h53m36.123745877s) [upload=196558]: Uploading segments... Aug 16 14:19:55.392524073 (188h53m36.123791712s) [upload=196558]: Got next segment. Inline: true Aug 16 14:19:55.392532609 (188h53m36.123800229s) [upload=196558]: Waiting for error group managing segments... Aug 16 14:19:55.392576782 (188h53m36.123844412s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:19:55.392583505 (188h53m36.123851135s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:19:55.392588043 (188h53m36.123855673s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:19:55.415388546 (188h53m36.146656196s) [upload=196558]: Done uploading segments... Aug 16 14:19:55.415406199 (188h53m36.146673849s) [upload=196558]: Upload finished. err: metaclient: Too Many Requests
Duration: 293.692556ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................X........X...............] duration:
1.990465954s
FAIL (2025-08-16 14:34:19 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.99048544s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...X...................................X....................] duration:
552.345856ms
FAIL (2025-08-16 14:34:19 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:19:55.392348603 (188h53m36.123616253s) [upload=196558]: Starting upload Aug 16 14:19:55.392413835 (188h53m36.123681465s) [upload=196558]: Done starting upload Aug 16 14:19:55.392478257 (188h53m36.123745877s) [upload=196558]: Uploading segments... Aug 16 14:19:55.392524073 (188h53m36.123791712s) [upload=196558]: Got next segment. Inline: true Aug 16 14:19:55.392532609 (188h53m36.123800229s) [upload=196558]: Waiting for error group managing segments... Aug 16 14:19:55.392576782 (188h53m36.123844412s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:19:55.392583505 (188h53m36.123851135s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:19:55.392588043 (188h53m36.123855673s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:19:55.415388546 (188h53m36.146656196s) [upload=196558]: Done uploading segments... Aug 16 14:19:55.415406199 (188h53m36.146673849s) [upload=196558]: Upload finished. err: metaclient: Too Many Requests
Duration: 552.366283ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.280287299s
FAIL (2025-08-16 14:33:58 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.280307107s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.842229083s
FAIL (2025-08-16 14:33:41 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 6.842268908s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................X........X..............] duration:
4.850911129s
FAIL (2025-08-16 14:33:25 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 4.850929393s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....X...................................X...................] duration:
1.046740148s
FAIL (2025-08-16 14:33:15 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:19:55.392348603 (188h53m36.123616253s) [upload=196558]: Starting upload Aug 16 14:19:55.392413835 (188h53m36.123681465s) [upload=196558]: Done starting upload Aug 16 14:19:55.392478257 (188h53m36.123745877s) [upload=196558]: Uploading segments... Aug 16 14:19:55.392524073 (188h53m36.123791712s) [upload=196558]: Got next segment. Inline: true Aug 16 14:19:55.392532609 (188h53m36.123800229s) [upload=196558]: Waiting for error group managing segments... Aug 16 14:19:55.392576782 (188h53m36.123844412s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:19:55.392583505 (188h53m36.123851135s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:19:55.392588043 (188h53m36.123855673s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:19:55.415388546 (188h53m36.146656196s) [upload=196558]: Done uploading segments... Aug 16 14:19:55.415406199 (188h53m36.146673849s) [upload=196558]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.046761508s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.59762707s
FAIL (2025-08-16 14:32:53 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.59764858s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....X...................................X..................] duration:
279.354191ms
FAIL (2025-08-16 14:32:15 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:19:55.392348603 (188h53m36.123616253s) [upload=196558]: Starting upload Aug 16 14:19:55.392413835 (188h53m36.123681465s) [upload=196558]: Done starting upload Aug 16 14:19:55.392478257 (188h53m36.123745877s) [upload=196558]: Uploading segments... Aug 16 14:19:55.392524073 (188h53m36.123791712s) [upload=196558]: Got next segment. Inline: true Aug 16 14:19:55.392532609 (188h53m36.123800229s) [upload=196558]: Waiting for error group managing segments... Aug 16 14:19:55.392576782 (188h53m36.123844412s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:19:55.392583505 (188h53m36.123851135s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:19:55.392588043 (188h53m36.123855673s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:19:55.415388546 (188h53m36.146656196s) [upload=196558]: Done uploading segments... Aug 16 14:19:55.415406199 (188h53m36.146673849s) [upload=196558]: Upload finished. err: metaclient: Too Many Requests
Duration: 279.368628ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X........X.............] duration:
1.931583479s
FAIL (2025-08-16 14:32:07 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.931602414s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
6.107605048s
FAIL (2025-08-16 14:32:07 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 6.107631668s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......X...................................X.................] duration:
183.166115ms
FAIL (2025-08-16 14:31:34 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:19:55.392348603 (188h53m36.123616253s) [upload=196558]: Starting upload Aug 16 14:19:55.392413835 (188h53m36.123681465s) [upload=196558]: Done starting upload Aug 16 14:19:55.392478257 (188h53m36.123745877s) [upload=196558]: Uploading segments... Aug 16 14:19:55.392524073 (188h53m36.123791712s) [upload=196558]: Got next segment. Inline: true Aug 16 14:19:55.392532609 (188h53m36.123800229s) [upload=196558]: Waiting for error group managing segments... Aug 16 14:19:55.392576782 (188h53m36.123844412s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:19:55.392583505 (188h53m36.123851135s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:19:55.392588043 (188h53m36.123855673s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:19:55.415388546 (188h53m36.146656196s) [upload=196558]: Done uploading segments... Aug 16 14:19:55.415406199 (188h53m36.146673849s) [upload=196558]: Upload finished. err: metaclient: Too Many Requests
Duration: 183.18449ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
5.155136463s
FAIL (2025-08-16 14:31:03 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.155157782s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................X........X............] duration:
3.011178476s
FAIL (2025-08-16 14:31:02 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 3.011202381s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......X...................................X................] duration:
406.57752ms
FAIL (2025-08-16 14:30:58 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:19:55.392348603 (188h53m36.123616253s) [upload=196558]: Starting upload Aug 16 14:19:55.392413835 (188h53m36.123681465s) [upload=196558]: Done starting upload Aug 16 14:19:55.392478257 (188h53m36.123745877s) [upload=196558]: Uploading segments... Aug 16 14:19:55.392524073 (188h53m36.123791712s) [upload=196558]: Got next segment. Inline: true Aug 16 14:19:55.392532609 (188h53m36.123800229s) [upload=196558]: Waiting for error group managing segments... Aug 16 14:19:55.392576782 (188h53m36.123844412s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:19:55.392583505 (188h53m36.123851135s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 16 14:19:55.392588043 (188h53m36.123855673s) [upload=196558]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 16 14:19:55.415388546 (188h53m36.146656196s) [upload=196558]: Done uploading segments... Aug 16 14:19:55.415406199 (188h53m36.146673849s) [upload=196558]: Upload finished. err: metaclient: Too Many Requests
Duration: 406.600574ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X........X...........] duration:
2.876945844s
FAIL (2025-08-16 14:30:41 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.876969418s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................................X........X..........] duration:
3.124344261s
FAIL (2025-08-16 14:30:10 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 16 14:23:10.478810879 (188h56m51.210078499s) [upload=196616]: Starting upload Aug 16 14:23:10.478872666 (188h56m51.210140316s) [upload=196616]: Done starting upload Aug 16 14:23:10.478932628 (188h56m51.210200259s) [upload=196616]: Uploading segments... Aug 16 14:23:10.479197186 (188h56m51.210464816s) [upload=196616]: Got next segment. Inline: false Aug 16 14:23:10.479224900 (188h56m51.21049252s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 16 14:23:10.479228587 (188h56m51.210496217s) [upload=196616]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 16 14:23:10.481835968 (188h56m51.213103598s) [upload=196616]: Next segment err: metaclient: Too Many Requests Aug 16 14:23:10.481847289 (188h56m51.213114909s) [upload=196616]: Done uploading segments... Aug 16 14:23:10.481872918 (188h56m51.213140538s) [upload=196616]: Upload finished. err: metaclient: Too Many Requests
Duration: 3.124373847s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.507086199s
FAIL (2025-08-16 14:30:08 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.50710306s
core
uplink
ap1
|