Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
143.986923ms
FAIL (2025-08-25 00:47:49 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 144.010397ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................XX..........................................] duration:
3.019463944s
FAIL (2025-08-25 00:47:20 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.01948257s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
2.035942466s
FAIL (2025-08-25 00:46:49 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.035963606s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
630.730593ms
FAIL (2025-08-25 00:46:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 630.743339ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
220.840664ms
FAIL (2025-08-25 00:46:41 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 220.86551ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................XX.........................................] duration:
3.225648939s
FAIL (2025-08-25 00:46:32 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.225669288s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................XX........................................] duration:
2.954017777s
FAIL (2025-08-25 00:46:09 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.954043226s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
380.288962ms
FAIL (2025-08-25 00:45:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 380.489048ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
1.338766277s
FAIL (2025-08-25 00:45:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.338785272s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
291.654655ms
FAIL (2025-08-25 00:45:37 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 291.68429ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
401.240518ms
FAIL (2025-08-25 00:45:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 401.261327ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................XX.......................................] duration:
3.12928447s
FAIL (2025-08-25 00:45:08 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.129310128s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
359.126114ms
FAIL (2025-08-25 00:45:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 359.146131ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................XX......................................] duration:
3.213131878s
FAIL (2025-08-25 00:44:48 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.213175089s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
147.851505ms
FAIL (2025-08-25 00:44:30 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 147.872125ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
684.799685ms
FAIL (2025-08-25 00:44:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 684.821066ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
586.141069ms
FAIL (2025-08-25 00:44:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 586.163762ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
138.086925ms
FAIL (2025-08-25 00:44:03 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 138.107073ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration:
3.197694804s
FAIL (2025-08-25 00:43:33 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.197720332s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
971.137999ms
FAIL (2025-08-25 00:43:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 971.151956ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
364.405ms
FAIL (2025-08-25 00:43:27 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 364.437723ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
572.498401ms
FAIL (2025-08-25 00:43:12 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 572.522216ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................XX....................................] duration:
3.855418049s
FAIL (2025-08-25 00:42:50 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.855446252s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
393.80286ms
FAIL (2025-08-25 00:42:39 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 393.836422ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
500.509084ms
FAIL (2025-08-25 00:42:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 500.521977ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
257.251031ms
FAIL (2025-08-25 00:42:30 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 257.275688ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.012836843s
FAIL (2025-08-25 00:42:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.01285609s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
151.426144ms
FAIL (2025-08-25 00:41:35 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 151.444399ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................XX...................................] duration:
2.980830092s
FAIL (2025-08-25 00:41:35 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 2.980858085s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
652.647373ms
FAIL (2025-08-25 00:41:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 652.666499ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.633415133s
FAIL (2025-08-25 00:41:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.63344003s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
257.702688ms
FAIL (2025-08-25 00:41:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 257.727084ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................XX..................................] duration:
3.311823898s
FAIL (2025-08-25 00:40:47 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.311847804s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
178.986698ms
FAIL (2025-08-25 00:40:32 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 179.00887ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
564.526318ms
FAIL (2025-08-25 00:40:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 564.544563ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
517.804284ms
FAIL (2025-08-25 00:40:07 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 517.833899ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................XX.................................] duration:
3.060953528s
FAIL (2025-08-25 00:40:06 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.060976952s
core
uplink
eu1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.286483846s
FAIL (2025-08-25 00:39:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1.286502852s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
356.474008ms
FAIL (2025-08-25 00:39:46 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 356.495929ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................XX................................] duration:
3.261797344s
FAIL (2025-08-25 00:39:11 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.261836778s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
204.748662ms
FAIL (2025-08-25 00:39:02 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 204.774081ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
397.232475ms
FAIL (2025-08-25 00:38:54 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 397.254276ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
521.424146ms
FAIL (2025-08-25 00:38:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 521.439905ms
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.728365759s
FAIL (2025-08-25 00:38:32 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.72839807s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
13.189321317s
FAIL (2025-08-25 00:38:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 13.189338108s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001213739s
FAIL (2025-08-25 00:38:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001230852s
edge
gateway
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.18291524s
FAIL (2025-08-25 00:37:58 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.182938343s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................XX...............................] duration:
3.478887727s
FAIL (2025-08-25 00:37:58 UTC)
orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded
Duration: 3.478908125s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
415.02954ms
FAIL (2025-08-25 00:37:27 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:25:28.400683968 (390h59m9.131951598s) [upload=406444]: Starting upload Aug 25 00:25:28.400740554 (390h59m9.132008184s) [upload=406444]: Done starting upload Aug 25 00:25:28.400969294 (390h59m9.132236924s) [upload=406444]: Uploading segments... Aug 25 00:25:28.400998699 (390h59m9.132266329s) [upload=406444]: Got next segment. Inline: false Aug 25 00:25:28.401020590 (390h59m9.13228822s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:25:28.401024117 (390h59m9.132291747s) [upload=406444]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 25 00:25:28.403715205 (390h59m9.134982825s) [upload=406444]: Next segment err: metaclient: Too Many Requests Aug 25 00:25:28.403727398 (390h59m9.134995017s) [upload=406444]: Done uploading segments... Aug 25 00:25:28.403753247 (390h59m9.135020877s) [upload=406444]: Upload finished. err: metaclient: Too Many Requests
Duration: 415.054678ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.20555126s
FAIL (2025-08-25 00:37:05 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 25 00:34:31.724244888 (391h8m12.455512539s) [upload=406603]: Starting upload Aug 25 00:34:31.724305743 (391h8m12.455573392s) [upload=406603]: Done starting upload Aug 25 00:34:31.724386896 (391h8m12.455654555s) [upload=406603]: Uploading segments... Aug 25 00:34:31.724479178 (391h8m12.455746829s) [upload=406603]: Got next segment. Inline: true Aug 25 00:34:31.724484819 (391h8m12.455752479s) [upload=406603]: Waiting for error group managing segments... Aug 25 00:34:31.724528791 (391h8m12.455796442s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 25 00:34:31.724533932 (391h8m12.455801581s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 25 00:34:31.724537368 (391h8m12.455805018s) [upload=406603]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 25 00:34:31.727506599 (391h8m12.458774249s) [upload=406603]: Done uploading segments... Aug 25 00:34:31.727519223 (391h8m12.458786872s) [upload=406603]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.205567511s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|