Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
5.111382998s
FAIL (2025-08-18 19:49:38 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.111403645s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [..................X...............X......................XX.] duration:
209.318358ms
FAIL (2025-08-18 19:49:37 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:48:45.889073605 (242h22m26.620341255s) [upload=252112]: Starting upload Aug 18 19:48:45.889148226 (242h22m26.620415876s) [upload=252112]: Done starting upload Aug 18 19:48:45.889226974 (242h22m26.620494624s) [upload=252112]: Uploading segments... Aug 18 19:48:45.889266378 (242h22m26.620534028s) [upload=252112]: Got next segment. Inline: true Aug 18 19:48:45.889271668 (242h22m26.620539318s) [upload=252112]: Waiting for error group managing segments... Aug 18 19:48:45.889338914 (242h22m26.620606564s) [upload=252112]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:48:45.889344815 (242h22m26.620612465s) [upload=252112]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:48:45.889348291 (242h22m26.620615952s) [upload=252112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:48:45.891482682 (242h22m26.622750301s) [upload=252112]: Done uploading segments... Aug 18 19:48:45.891494854 (242h22m26.622762484s) [upload=252112]: Upload finished. err: metaclient: Too Many Requests
Duration: 209.352603ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................X....X.............................] duration:
576.484696ms
FAIL (2025-08-18 19:49:10 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 576.506458ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
4.989189404s
FAIL (2025-08-18 19:48:56 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.989210293s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
4 out of 60 is failed (threshold=3) [...................X...............X......................XX] duration:
15.45727ms
FAIL (2025-08-18 19:48:45 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:48:45.889073605 (242h22m26.620341255s) [upload=252112]: Starting upload Aug 18 19:48:45.889148226 (242h22m26.620415876s) [upload=252112]: Done starting upload Aug 18 19:48:45.889226974 (242h22m26.620494624s) [upload=252112]: Uploading segments... Aug 18 19:48:45.889266378 (242h22m26.620534028s) [upload=252112]: Got next segment. Inline: true Aug 18 19:48:45.889271668 (242h22m26.620539318s) [upload=252112]: Waiting for error group managing segments... Aug 18 19:48:45.889338914 (242h22m26.620606564s) [upload=252112]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:48:45.889344815 (242h22m26.620612465s) [upload=252112]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:48:45.889348291 (242h22m26.620615952s) [upload=252112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:48:45.891482682 (242h22m26.622750301s) [upload=252112]: Done uploading segments... Aug 18 19:48:45.891494854 (242h22m26.622762484s) [upload=252112]: Upload finished. err: metaclient: Too Many Requests
Duration: 15.4731ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
4.960156655s
FAIL (2025-08-18 19:48:27 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.960184638s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
3 out of 60 is failed (threshold=3) [....................X...............X......................X] duration:
130.931839ms
FAIL (2025-08-18 19:48:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: invalid range 0:1024 (size:1024) ======LOGS====== Aug 18 19:48:20.665747054 (242h22m1.397014703s) [upload=252103]: Starting upload Aug 18 19:48:20.665799272 (242h22m1.397066921s) [upload=252103]: Done starting upload Aug 18 19:48:20.665875956 (242h22m1.397143596s) [upload=252103]: Uploading segments... Aug 18 19:48:20.665929316 (242h22m1.397196966s) [upload=252103]: Got next segment. Inline: true Aug 18 19:48:20.665937171 (242h22m1.397204821s) [upload=252103]: Waiting for error group managing segments... Aug 18 19:48:20.665987395 (242h22m1.397255045s) [upload=252103]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:48:20.665992525 (242h22m1.397260174s) [upload=252103]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:48:20.665996162 (242h22m1.397263811s) [upload=252103]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:48:20.737208221 (242h22m1.468475871s) [upload=252103]: Done uploading segments... Aug 18 19:48:20.737239590 (242h22m1.46850724s) [upload=252103]: Upload finished. err: <nil>
Duration: 130.961214ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................X....X............................] duration:
778.460971ms
FAIL (2025-08-18 19:48:17 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 778.487351ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
4.795834918s
FAIL (2025-08-18 19:47:58 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.79586214s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................X....X...........................] duration:
568.221111ms
FAIL (2025-08-18 19:47:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 568.240468ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....................X...............X......................] duration:
232.804834ms
FAIL (2025-08-18 19:47:09 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 232.830052ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................X....X..........................] duration:
434.316796ms
FAIL (2025-08-18 19:46:49 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 434.35125ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
4.898896855s
FAIL (2025-08-18 19:46:49 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.898922082s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................X...............X.....................] duration:
138.365745ms
FAIL (2025-08-18 19:46:24 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 138.388888ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................X....X.........................] duration:
915.564916ms
FAIL (2025-08-18 19:45:56 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 915.584893ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................X...............X....................] duration:
381.833673ms
FAIL (2025-08-18 19:45:41 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 381.857757ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
4.861309238s
FAIL (2025-08-18 19:45:37 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.861331219s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................X....X........................] duration:
656.878858ms
FAIL (2025-08-18 19:45:25 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 656.945483ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
4.833506199s
FAIL (2025-08-18 19:45:16 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.833532018s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........................X...............X...................] duration:
228.667948ms
FAIL (2025-08-18 19:45:07 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 228.687987ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............................X....X.......................] duration:
1.076095194s
FAIL (2025-08-18 19:45:02 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.076110703s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [................................X....X......................] duration:
706.198686ms
FAIL (2025-08-18 19:44:42 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 706.225347ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
5.287115895s
FAIL (2025-08-18 19:44:22 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.287140982s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........................X...............X..................] duration:
398.084005ms
FAIL (2025-08-18 19:43:53 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 398.104244ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................................X....X.....................] duration:
492.422504ms
FAIL (2025-08-18 19:43:44 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 492.447671ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
4.937957876s
FAIL (2025-08-18 19:43:39 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.93800712s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................................X....X....................] duration:
463.686845ms
FAIL (2025-08-18 19:43:18 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 463.70558ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........................X...............X.................] duration:
288.342826ms
FAIL (2025-08-18 19:43:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 288.360059ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.030609569s
FAIL (2025-08-18 19:42:52 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.030633664s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................................X....X...................] duration:
720.585825ms
FAIL (2025-08-18 19:42:36 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 720.621603ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...........................X...............X................] duration:
164.562763ms
FAIL (2025-08-18 19:42:28 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 164.578343ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
5.29782855s
FAIL (2025-08-18 19:42:24 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.2978494s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
4.91720905s
FAIL (2025-08-18 19:42:01 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.917232834s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [............................X...............X...............] duration:
148.247141ms
FAIL (2025-08-18 19:41:39 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 148.26759ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................................X....X..................] duration:
457.793385ms
FAIL (2025-08-18 19:41:38 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 457.817079ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............................X...............X..............] duration:
214.607918ms
FAIL (2025-08-18 19:41:13 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 214.626732ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
5.25356872s
FAIL (2025-08-18 19:41:06 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.253596262s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X....X.................] duration:
457.24358ms
FAIL (2025-08-18 19:40:31 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 457.265181ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
4.904177741s
FAIL (2025-08-18 19:40:15 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.904205113s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............................X...............X.............] duration:
114.271818ms
FAIL (2025-08-18 19:40:06 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 114.291114ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
4.919372872s
FAIL (2025-08-18 19:39:33 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.919397849s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................................X....X................] duration:
485.165959ms
FAIL (2025-08-18 19:39:32 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 485.219441ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X....X...............] duration:
816.424825ms
FAIL (2025-08-18 19:39:16 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 816.447848ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............................X...............X............] duration:
170.967917ms
FAIL (2025-08-18 19:39:00 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 170.982265ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
4.954999297s
FAIL (2025-08-18 19:38:49 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 4.955018293s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................................X....X..............] duration:
497.767183ms
FAIL (2025-08-18 19:38:40 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 497.794545ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................................X...............X...........] duration:
133.872794ms
FAIL (2025-08-18 19:38:29 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 133.893112ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X....X.............] duration:
734.569789ms
FAIL (2025-08-18 19:38:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:27:32.511093581 (242h1m13.242361211s) [upload=251742]: Starting upload Aug 18 19:27:32.511142133 (242h1m13.242409763s) [upload=251742]: Done starting upload Aug 18 19:27:32.511287967 (242h1m13.242555597s) [upload=251742]: Uploading segments... Aug 18 19:27:32.511379579 (242h1m13.242647209s) [upload=251742]: Got next segment. Inline: false Aug 18 19:27:32.511430944 (242h1m13.242698565s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:27:32.511437096 (242h1m13.242704726s) [upload=251742]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 18 19:27:32.514566214 (242h1m13.245833844s) [upload=251742]: Next segment err: metaclient: Too Many Requests Aug 18 19:27:32.514576373 (242h1m13.245844003s) [upload=251742]: Done uploading segments... Aug 18 19:27:32.514585009 (242h1m13.245852639s) [upload=251742]: Upload finished. err: metaclient: Too Many Requests
Duration: 734.596258ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.062365831s
FAIL (2025-08-18 19:37:47 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.062400606s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................................X...............X..........] duration:
133.941985ms
FAIL (2025-08-18 19:37:44 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 18 19:32:04.150795103 (242h5m44.882062752s) [upload=251816]: Starting upload Aug 18 19:32:04.150971773 (242h5m44.882239424s) [upload=251816]: Done starting upload Aug 18 19:32:04.151100085 (242h5m44.882367746s) [upload=251816]: Uploading segments... Aug 18 19:32:04.151159036 (242h5m44.882426696s) [upload=251816]: Got next segment. Inline: true Aug 18 19:32:04.151166620 (242h5m44.88243427s) [upload=251816]: Waiting for error group managing segments... Aug 18 19:32:04.151227063 (242h5m44.882494723s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 18 19:32:04.151235009 (242h5m44.882502658s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 18 19:32:04.151240087 (242h5m44.882507738s) [upload=251816]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 18 19:32:04.153992839 (242h5m44.885260499s) [upload=251816]: Done uploading segments... Aug 18 19:32:04.154012065 (242h5m44.885279715s) [upload=251816]: Upload finished. err: metaclient: Too Many Requests
Duration: 133.972813ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|