Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
8.870939079s
FAIL (2025-08-22 12:58:41 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 8.870967414s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.............................................XX.............] duration:
184.189761ms
FAIL (2025-08-22 12:58:18 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 184.222782ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................X......................................X] duration:
17.715877ms
FAIL (2025-08-22 12:58:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:58:04.454745863 (331h31m45.186013493s) [upload=344671]: Starting upload Aug 22 12:58:04.454786480 (331h31m45.18605411s) [upload=344671]: Done starting upload Aug 22 12:58:04.454860990 (331h31m45.18612862s) [upload=344671]: Uploading segments... Aug 22 12:58:04.455003919 (331h31m45.186271539s) [upload=344671]: Got next segment. Inline: false Aug 22 12:58:04.455045448 (331h31m45.186313068s) [upload=344671]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:58:04.455049105 (331h31m45.186316725s) [upload=344671]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:58:04.459191524 (331h31m45.190459154s) [upload=344671]: Next segment err: metaclient: Too Many Requests Aug 22 12:58:04.459202625 (331h31m45.190470255s) [upload=344671]: Done uploading segments... Aug 22 12:58:04.459211321 (331h31m45.190478951s) [upload=344671]: Upload finished. err: metaclient: Too Many Requests
Duration: 17.731346ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
6.647631339s
FAIL (2025-08-22 12:57:46 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 6.647656868s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..............................................XX............] duration:
405.436594ms
FAIL (2025-08-22 12:57:45 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 405.46137ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
1.123917395s
FAIL (2025-08-22 12:56:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.123939989s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.620066349s
FAIL (2025-08-22 12:56:38 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.620092008s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...............................................XX...........] duration:
433.876365ms
FAIL (2025-08-22 12:56:30 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 433.893727ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.993785061s
FAIL (2025-08-22 12:56:03 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.993803887s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
2.309981394s
FAIL (2025-08-22 12:55:49 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.310003916s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [................................................XX..........] duration:
154.028823ms
FAIL (2025-08-22 12:55:29 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 154.051196ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
1.269241741s
FAIL (2025-08-22 12:55:15 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.269272187s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
7.690467022s
FAIL (2025-08-22 12:54:58 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 7.690488493s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.................................................XX.........] duration:
1.021325163s
FAIL (2025-08-22 12:54:28 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 1.021338508s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
445.349526ms
FAIL (2025-08-22 12:54:12 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 445.36749ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
7.097797485s
FAIL (2025-08-22 12:53:56 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 7.098031304s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..................................................XX........] duration:
227.836879ms
FAIL (2025-08-22 12:53:17 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 227.857037ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
911.602634ms
FAIL (2025-08-22 12:53:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 911.620928ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
7.177960415s
FAIL (2025-08-22 12:52:47 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 7.17798481s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [...................................................XX.......] duration:
160.874485ms
FAIL (2025-08-22 12:52:35 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 160.901816ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
8.786218284s
FAIL (2025-08-22 12:52:33 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 8.786259152s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [....................................................XX......] duration:
373.102594ms
FAIL (2025-08-22 12:52:07 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 373.1223ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
436.876322ms
FAIL (2025-08-22 12:51:55 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 436.900867ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration:
1.058754788s
FAIL (2025-08-22 12:51:30 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 1.058781339s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.309389796s
FAIL (2025-08-22 12:51:28 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.30941307s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
1.661743645s
FAIL (2025-08-22 12:51:25 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.661774172s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
1.975967497s
FAIL (2025-08-22 12:51:09 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.975986372s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
6.559149866s
FAIL (2025-08-22 12:50:49 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 6.559176496s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
498.572372ms
FAIL (2025-08-22 12:50:27 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 498.598541ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [......................................................XX....] duration:
149.524642ms
FAIL (2025-08-22 12:50:16 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 149.548577ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
8.466676892s
FAIL (2025-08-22 12:49:43 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 8.466703933s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
978.763613ms
FAIL (2025-08-22 12:49:35 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 978.792808ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................XX...] duration:
192.091393ms
FAIL (2025-08-22 12:49:23 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 192.113965ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
5.782121558s
FAIL (2025-08-22 12:48:55 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.782146996s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [........................................................XX..] duration:
345.44322ms
FAIL (2025-08-22 12:48:53 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 345.458669ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
501.688508ms
FAIL (2025-08-22 12:48:27 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 501.711751ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [.........................................................XX.] duration:
251.121744ms
FAIL (2025-08-22 12:48:22 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 251.153163ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
1.339371086s
FAIL (2025-08-22 12:47:57 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.33939404s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
14.658939371s
FAIL (2025-08-22 12:47:54 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 14.658956093s
core
uplink
ap1
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
12.400586569s
FAIL (2025-08-22 12:47:32 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 12.400609652s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
957.203794ms
FAIL (2025-08-22 12:47:21 UTC)
orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/1K ======LOGS====== Aug 22 12:47:20.501277811 (331h21m1.232545441s) [upload=344489]: Starting upload Aug 22 12:47:20.501322064 (331h21m1.232589694s) [upload=344489]: Done starting upload Aug 22 12:47:20.501374874 (331h21m1.232642504s) [upload=344489]: Uploading segments... Aug 22 12:47:20.501400582 (331h21m1.232668212s) [upload=344489]: Got next segment. Inline: true Aug 22 12:47:20.501404289 (331h21m1.232671919s) [upload=344489]: Waiting for error group managing segments... Aug 22 12:47:20.501434876 (331h21m1.232702507s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:47:20.501438293 (331h21m1.232705923s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:47:20.501440667 (331h21m1.232708287s) [upload=344489]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:47:21.405382175 (331h21m2.136649805s) [upload=344489]: Done uploading segments... Aug 22 12:47:21.405395971 (331h21m2.136663601s) [upload=344489]: Upload finished. err: <nil>
Duration: 957.22722ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
2.102656503s
FAIL (2025-08-22 12:46:48 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 2.102681329s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
5.610993789s
FAIL (2025-08-22 12:46:32 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 5.611012924s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
1.749279203s
FAIL (2025-08-22 12:46:20 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.749295634s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
16.223026ms
FAIL (2025-08-22 12:46:09 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:46:09.828201876 (331h19m50.559469527s) [upload=344468]: Starting upload Aug 22 12:46:09.828279152 (331h19m50.559546803s) [upload=344468]: Done starting upload Aug 22 12:46:09.828344515 (331h19m50.559612165s) [upload=344468]: Uploading segments... Aug 22 12:46:09.828394560 (331h19m50.5596622s) [upload=344468]: Got next segment. Inline: true Aug 22 12:46:09.828400491 (331h19m50.559668131s) [upload=344468]: Waiting for error group managing segments... Aug 22 12:46:09.828443272 (331h19m50.559710921s) [upload=344468]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:46:09.828448692 (331h19m50.559716331s) [upload=344468]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 22 12:46:09.828452008 (331h19m50.559719658s) [upload=344468]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 22 12:46:09.831349944 (331h19m50.562617604s) [upload=344468]: Done uploading segments... Aug 22 12:46:09.831364863 (331h19m50.562632523s) [upload=344468]: Upload finished. err: metaclient: Too Many Requests
Duration: 16.246561ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
538.136158ms
FAIL (2025-08-22 12:45:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 538.178217ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
6.559334386s
FAIL (2025-08-22 12:45:32 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 6.559355585s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.44215325s
FAIL (2025-08-22 12:45:14 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.442182335s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "121RTSDpyNZVcEU84Ticf2L1ntiuUimbWgfATz21tuvgk3vzoA6@ap1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.ap1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
8.1110035s
FAIL (2025-08-22 12:44:54 UTC)
orbiter warning: operation: DownloadObject uplink: metaclient: context deadline exceeded
Duration: 8.111021223s
core
uplink
ap1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
370.884955ms
FAIL (2025-08-22 12:44:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: too many requests ======LOGS====== Aug 22 12:27:34.547441694 (331h1m15.278709314s) [upload=344147]: Starting upload Aug 22 12:27:34.547482662 (331h1m15.278750292s) [upload=344147]: Done starting upload Aug 22 12:27:34.547563053 (331h1m15.278830683s) [upload=344147]: Uploading segments... Aug 22 12:27:34.547701333 (331h1m15.278968963s) [upload=344147]: Got next segment. Inline: false Aug 22 12:27:34.547731560 (331h1m15.27899919s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 22 12:27:34.547735378 (331h1m15.279003008s) [upload=344147]: Flush batch item: *pb.BatchRequestItem_SegmentBegin Aug 22 12:27:34.551117328 (331h1m15.282384958s) [upload=344147]: Next segment err: metaclient: Too Many Requests Aug 22 12:27:34.551133488 (331h1m15.282401118s) [upload=344147]: Done uploading segments... Aug 22 12:27:34.551142895 (331h1m15.282410526s) [upload=344147]: Upload finished. err: metaclient: Too Many Requests
Duration: 370.910834ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|