Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
129.193386ms
FAIL (2025-08-08 07:41:05 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests
Duration: 129.216378ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
139.185929ms
FAIL (2025-08-08 07:40:24 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests
Duration: 139.215744ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
151.671896ms
FAIL (2025-08-08 07:39:19 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests
Duration: 151.760251ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
17.816147ms
FAIL (2025-08-08 07:38:33 UTC)
orbiter warning: operation: Commit ======ERROR====== uplink: too many requests ======LOGS====== 1:24.380223643 (15h4m37.967279854s) [upload=15580]: Done uploading segments... Aug 8 07:31:24.380235495 (15h4m37.967291706s) [upload=15580]: Upload finished. err: <nil> Aug 8 07:32:31.769309432 (15h5m45.356365673s) [upload=15599]: Starting upload Aug 8 07:32:31.769358563 (15h5m45.356414804s) [upload=15599]: Done starting upload Aug 8 07:32:31.769427131 (15h5m45.356483373s) [upload=15599]: Uploading segments... Aug 8 07:32:31.769461335 (15h5m45.356517576s) [upload=15599]: Got next segment. Inline: true Aug 8 07:32:31.769466534 (15h5m45.356522776s) [upload=15599]: Waiting for error group managing segments... Aug 8 07:32:31.769507311 (15h5m45.356563552s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:32:31.769512431 (15h5m45.356568671s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:32:31.769515687 (15h5m45.356571917s) [upload=15599]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:32:31.821109527 (15h5m45.408165778s) [upload=15599]: Done uploading segments... Aug 8 07:32:31.821138460 (15h5m45.408194731s) [upload=15599]: Upload finished. err: <nil> Aug 8 07:33:21.342417331 (15h6m34.929473572s) [upload=15614]: Starting upload Aug 8 07:33:21.342485929 (15h6m34.92954217s) [upload=15614]: Done starting upload Aug 8 07:33:21.342553876 (15h6m34.929610107s) [upload=15614]: Uploading segments... Aug 8 07:33:21.342590324 (15h6m34.929646565s) [upload=15614]: Got next segment. Inline: true Aug 8 07:33:21.342595904 (15h6m34.929652145s) [upload=15614]: Waiting for error group managing segments... Aug 8 07:33:21.342637332 (15h6m34.929693563s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:33:21.342642261 (15h6m34.929698503s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:21.342645677 (15h6m34.929701919s) [upload=15614]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:33:21.406200486 (15h6m34.993256696s) [upload=15614]: Done uploading segments... Aug 8 07:33:21.406214111 (15h6m34.993270332s) [upload=15614]: Upload finished. err: <nil> Aug 8 07:33:49.938059079 (15h7m3.525115321s) [upload=15623]: Starting upload Aug 8 07:33:49.938125413 (15h7m3.525181654s) [upload=15623]: Done starting upload Aug 8 07:33:49.938197296 (15h7m3.525253538s) [upload=15623]: Uploading segments... Aug 8 07:33:49.938232693 (15h7m3.525288934s) [upload=15623]: Got next segment. Inline: true Aug 8 07:33:49.938238764 (15h7m3.525295006s) [upload=15623]: Waiting for error group managing segments... Aug 8 07:33:49.938281404 (15h7m3.525337646s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:33:49.938287165 (15h7m3.525343406s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:33:49.938290742 (15h7m3.525346983s) [upload=15623]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:33:50.134165957 (15h7m3.721222208s) [upload=15623]: Done uploading segments... Aug 8 07:33:50.134184350 (15h7m3.721240602s) [upload=15623]: Upload finished. err: <nil> Aug 8 07:34:50.960424921 (15h8m4.547481162s) [upload=15644]: Starting upload Aug 8 07:34:50.960491324 (15h8m4.547547565s) [upload=15644]: Done starting upload Aug 8 07:34:50.960563439 (15h8m4.547619681s) [upload=15644]: Uploading segments... Aug 8 07:34:50.960608603 (15h8m4.547664854s) [upload=15644]: Got next segment. Inline: true Aug 8 07:34:50.960614675 (15h8m4.547670915s) [upload=15644]: Waiting for error group managing segments... Aug 8 07:34:50.960662715 (15h8m4.547718956s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:34:50.960669036 (15h8m4.547725278s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:34:50.960672843 (15h8m4.547729085s) [upload=15644]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:34:51.015446971 (15h8m4.602503212s) [upload=15644]: Done uploading segments... Aug 8 07:34:51.015467048 (15h8m4.60252329s) [upload=15644]: Upload finished. err: <nil> Aug 8 07:35:19.183757296 (15h8m32.770813538s) [upload=15649]: Starting upload Aug 8 07:35:19.183838608 (15h8m32.770894859s) [upload=15649]: Done starting upload Aug 8 07:35:19.183954104 (15h8m32.771010345s) [upload=15649]: Uploading segments... Aug 8 07:35:19.183996463 (15h8m32.771052704s) [upload=15649]: Got next segment. Inline: true Aug 8 07:35:19.184002284 (15h8m32.771058535s) [upload=15649]: Waiting for error group managing segments... Aug 8 07:35:19.184044383 (15h8m32.771100625s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:35:19.184049663 (15h8m32.771105904s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:35:19.184053169 (15h8m32.771109411s) [upload=15649]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:35:19.243291931 (15h8m32.830348182s) [upload=15649]: Done uploading segments... Aug 8 07:35:19.243307820 (15h8m32.830364062s) [upload=15649]: Upload finished. err: <nil> Aug 8 07:36:19.333356281 (15h9m32.920412513s) [upload=15668]: Starting upload Aug 8 07:36:19.333424730 (15h9m32.920480961s) [upload=15668]: Done starting upload Aug 8 07:36:19.333495762 (15h9m32.920551983s) [upload=15668]: Uploading segments... Aug 8 07:36:19.333539955 (15h9m32.920596196s) [upload=15668]: Got next segment. Inline: true Aug 8 07:36:19.333546026 (15h9m32.920602257s) [upload=15668]: Waiting for error group managing segments... Aug 8 07:36:19.333593585 (15h9m32.920649816s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:19.333599806 (15h9m32.920656037s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:19.333604235 (15h9m32.920660466s) [upload=15668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:19.390544896 (15h9m32.977601137s) [upload=15668]: Done uploading segments... Aug 8 07:36:19.390558833 (15h9m32.977615073s) [upload=15668]: Upload finished. err: <nil> Aug 8 07:36:37.512921106 (15h9m51.099977348s) [upload=15674]: Starting upload Aug 8 07:36:37.513013459 (15h9m51.100069699s) [upload=15674]: Done starting upload Aug 8 07:36:37.513079943 (15h9m51.100136184s) [upload=15674]: Uploading segments... Aug 8 07:36:37.513131830 (15h9m51.100188071s) [upload=15674]: Got next segment. Inline: true Aug 8 07:36:37.513137881 (15h9m51.100194122s) [upload=15674]: Waiting for error group managing segments... Aug 8 07:36:37.513184869 (15h9m51.100241111s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:37.513191010 (15h9m51.100247252s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:37.513194737 (15h9m51.100250979s) [upload=15674]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:37.565464841 (15h9m51.152521082s) [upload=15674]: Done uploading segments... Aug 8 07:36:37.565482684 (15h9m51.152538925s) [upload=15674]: Upload finished. err: <nil> Aug 8 07:36:58.920843823 (15h10m12.507900045s) [upload=15685]: Starting upload Aug 8 07:36:58.920921889 (15h10m12.5079781s) [upload=15685]: Done starting upload Aug 8 07:36:58.921014462 (15h10m12.508070674s) [upload=15685]: Uploading segments... Aug 8 07:36:58.921051942 (15h10m12.508108153s) [upload=15685]: Got next segment. Inline: true Aug 8 07:36:58.921056751 (15h10m12.508112962s) [upload=15685]: Waiting for error group managing segments... Aug 8 07:36:58.921095153 (15h10m12.508151364s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:36:58.921100823 (15h10m12.508157035s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:36:58.921104460 (15h10m12.508160672s) [upload=15685]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:36:58.974322112 (15h10m12.561378323s) [upload=15685]: Done uploading segments... Aug 8 07:36:58.974335476 (15h10m12.561391688s) [upload=15685]: Upload finished. err: <nil> Aug 8 07:37:18.549598640 (15h10m32.136654881s) [upload=15688]: Starting upload Aug 8 07:37:18.549652862 (15h10m32.136709103s) [upload=15688]: Done starting upload Aug 8 07:37:18.549725086 (15h10m32.136781327s) [upload=15688]: Uploading segments... Aug 8 07:37:18.549771754 (15h10m32.136827995s) [upload=15688]: Got next segment. Inline: true Aug 8 07:37:18.549777335 (15h10m32.136833575s) [upload=15688]: Waiting for error group managing segments... Aug 8 07:37:18.549825304 (15h10m32.136881544s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:37:18.549830874 (15h10m32.136887125s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:37:18.549834401 (15h10m32.136890651s) [upload=15688]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:37:18.618303321 (15h10m32.205359563s) [upload=15688]: Done uploading segments... Aug 8 07:37:18.618325323 (15h10m32.205381564s) [upload=15688]: Upload finished. err: <nil> Aug 8 07:38:33.515688106 (15h11m47.102744348s) [upload=15711]: Starting upload Aug 8 07:38:33.515754800 (15h11m47.102811042s) [upload=15711]: Done starting upload Aug 8 07:38:33.515831273 (15h11m47.102887524s) [upload=15711]: Uploading segments... Aug 8 07:38:33.515961987 (15h11m47.103018238s) [upload=15711]: Got next segment. Inline: true Aug 8 07:38:33.515970483 (15h11m47.103026734s) [upload=15711]: Waiting for error group managing segments... Aug 8 07:38:33.516031648 (15h11m47.103087889s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Aug 8 07:38:33.516038049 (15h11m47.103094291s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Aug 8 07:38:33.516041546 (15h11m47.103097787s) [upload=15711]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Aug 8 07:38:33.518697103 (15h11m47.105753344s) [upload=15711]: Done uploading segments... Aug 8 07:38:33.518714725 (15h11m47.105770967s) [upload=15711]: Upload finished. err: metaclient: Too Many Requests
Duration: 17.840111ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
374.65218ms
FAIL (2025-08-08 07:18:31 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 374.67914ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
495.564492ms
FAIL (2025-08-08 07:17:21 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 495.596843ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
421.608537ms
FAIL (2025-08-08 07:16:46 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 421.629336ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
368.908047ms
FAIL (2025-08-08 07:16:04 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 368.930891ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
509.18164ms
FAIL (2025-08-08 07:15:11 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 509.209372ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
551.072633ms
FAIL (2025-08-08 07:14:50 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 551.110042ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
727.246331ms
FAIL (2025-08-08 07:13:51 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 727.274613ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
359.815415ms
FAIL (2025-08-08 07:12:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 359.833728ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
407.474372ms
FAIL (2025-08-08 07:12:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 407.496584ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
570.687709ms
FAIL (2025-08-08 07:10:51 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 570.712466ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
332.105386ms
FAIL (2025-08-08 07:10:14 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 332.127456ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
674.658907ms
FAIL (2025-08-08 07:09:31 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 674.680507ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
361.908234ms
FAIL (2025-08-08 07:09:04 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 361.967004ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
389.428725ms
FAIL (2025-08-08 07:08:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 389.451778ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
389.811251ms
FAIL (2025-08-08 07:07:34 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 389.833834ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
385.061286ms
FAIL (2025-08-08 07:06:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 385.081555ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
321.980487ms
FAIL (2025-08-08 07:06:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 322.001187ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
393.513254ms
FAIL (2025-08-08 07:05:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 393.616646ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
375.723614ms
FAIL (2025-08-08 07:05:20 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 375.756625ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
358.635946ms
FAIL (2025-08-08 07:04:40 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 358.65982ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
391.255499ms
FAIL (2025-08-08 07:04:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 391.281467ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
386.785345ms
FAIL (2025-08-08 07:03:42 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 386.818386ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
369.757097ms
FAIL (2025-08-08 07:02:41 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 369.785981ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
365.946281ms
FAIL (2025-08-08 07:01:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 365.96731ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
353.338917ms
FAIL (2025-08-08 07:00:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 353.365246ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
365.871002ms
FAIL (2025-08-08 06:59:43 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 365.89085ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
438.300728ms
FAIL (2025-08-08 06:59:21 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 438.327889ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
367.783833ms
FAIL (2025-08-08 06:58:32 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 367.806024ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
544.02424ms
FAIL (2025-08-08 06:57:46 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 544.054596ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
618.084313ms
FAIL (2025-08-08 06:57:02 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 618.108258ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
385.26768ms
FAIL (2025-08-08 06:56:08 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 385.282519ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
387.477485ms
FAIL (2025-08-08 06:55:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 387.509363ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
818.344227ms
FAIL (2025-08-08 06:54:49 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 818.362461ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
374.394464ms
FAIL (2025-08-08 06:53:48 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 374.442164ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
404.501707ms
FAIL (2025-08-08 06:52:47 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 404.520653ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
372.998126ms
FAIL (2025-08-08 06:52:18 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 373.031057ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
416.295281ms
FAIL (2025-08-08 06:51:25 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 416.320729ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
605.417646ms
FAIL (2025-08-08 06:50:41 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 605.470223ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
363.910601ms
FAIL (2025-08-08 06:50:25 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 363.932272ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
342.41291ms
FAIL (2025-08-08 06:49:19 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 342.453785ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
429.966909ms
FAIL (2025-08-08 06:48:07 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 429.988539ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
332.78202ms
FAIL (2025-08-08 06:47:39 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 332.807307ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
453.286831ms
FAIL (2025-08-08 06:47:06 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 453.314504ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
444.754061ms
FAIL (2025-08-08 06:46:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 444.782603ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
380.448966ms
FAIL (2025-08-08 06:45:14 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 380.476306ms
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.051705602s
FAIL (2025-08-08 06:44:50 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: too many requests
======LOGS======
XOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065025521 (14h5m53.652081792s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065205998 (14h5m53.652262239s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065455032 (14h5m53.652511273s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065786381 (14h5m53.652842622s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.065845953 (14h5m53.652902193s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066263101 (14h5m53.653319342s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066327521 (14h5m53.653383763s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.066919006 (14h5m53.653975247s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067254953 (14h5m53.654311194s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067466127 (14h5m53.654522368s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067873497 (14h5m53.654929738s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.067987530 (14h5m53.655043771s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068356108 (14h5m53.655412349s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068429946 (14h5m53.655486187s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.068896717 (14h5m53.655952958s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.069016020 (14h5m53.656072261s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Uploading piece...
Aug 8 06:32:40.225413177 (14h5m53.812469418s) [upload=14568]: Next returned nil segment
Aug 8 06:32:40.225429067 (14h5m53.812485298s) [upload=14568]: Waiting for error group managing segments...
Aug 8 06:32:40.238535474 (14h5m53.825591705s) [noise=true , address=104.36.82.61:4404 , piece_id=DF4W6VI4HG7XX444VNPBKGDLK4EYJQZ5SKCZGBV6UNPJGGFA6BEA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.239293129 (14h5m53.826349371s) [noise=true , address=104.36.82.42:4808 , piece_id=5AGVDHXGAVLF6OPWT4KYTW6TFAPDPNADOPZ5C3KXAXRLI4OUNBDQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.241689332 (14h5m53.828745574s) [noise=true , address=104.36.82.23:4457 , piece_id=OOK6SCTGDGEMBOCXSBR2RC5IUS2EYPANIISO2IAPXNRWUFA4ORMQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242096392 (14h5m53.829152623s) [noise=true , address=104.36.82.45:4938 , piece_id=HYAUM7LFB7YKMLPBHD7PEBP6Y54JMZ2UAJTDTBCP3JAB3IBJHJEQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242267271 (14h5m53.829323512s) [noise=true , address=104.36.82.58:4273 , piece_id=H6VG6QUSKVNBAJRDQUV4PZIKBFGAM7OIX7BTB4UO5YW6EHVFR2FQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.242818750 (14h5m53.829874981s) [noise=true , address=104.36.82.37:4657 , piece_id=DDJRHIGSD66WQQ3SAPQRKC7RHTHXASXDVNMIKQQDRTWKK5ADIUOQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.243569251 (14h5m53.830625473s) [noise=true , address=104.36.82.38:4693 , piece_id=QPR6POCGGHYGULCXPEOZBBSDBOV5DLTIW6ZUKMGPIJZLB3F3HNJA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244340481 (14h5m53.831396702s) [noise=true , address=104.36.82.60:4359 , piece_id=PT3LJH7ROPW2UXME5HDMDAHIFPZ3JILDNT6LUTNHYDSNFRRGIR4A, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.244984754 (14h5m53.832040985s) [noise=true , address=104.36.82.43:4854 , piece_id=3YYPCYCMQQSKEG5WE6PD3QF5XJI3GN3FIAUNBQDOQ2URBGMHD7AA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.245524792 (14h5m53.832581023s) [noise=true , address=104.36.82.37:4684 , piece_id=PV6SS66AY2YJEC2CBYLQ4LQLWUS3KWVPIPPN46OAAFC4ZZQGX5XQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.246872779 (14h5m53.83392901s) [noise=true , address=104.36.82.36:4641 , piece_id=R7R33QBSBJUWWKVI2UXEDRKDCOCVYKMYVE2CEWK2MATSBZ3MGJIQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247331215 (14h5m53.834387456s) [noise=true , address=104.36.82.34:4561 , piece_id=D42SM3LJAAC7HHZWQEYBPWT6NOUQW3VUVHNPAODKKY54CWZWDQBQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.247891481 (14h5m53.834947722s) [noise=true , address=104.36.82.62:4439 , piece_id=CMLMZQQLTVF3KHEIHNQTZAELFG3465HOYKMCK2UDYOQHPROML3SA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248431108 (14h5m53.835487338s) [noise=true , address=104.36.82.13:4204 , piece_id=WT2WD2ZAU5MBXXNBZFXLOVXMWUFOMPUXOBTOA7P67NE2JIELPZWQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: <nil>
Aug 8 06:32:40.248441287 (14h5m53.835497528s) [seg_pos={0 0}, upload=14568]: Segment reached optimal threshold of 14 pieces.
Aug 8 06:32:40.248563916 (14h5m53.835620157s) [noise=true , address=104.36.82.59:4319 , piece_id=AIU65US456ESICJU42TGN43EC7KRVDXAQA4TXF3LEJ66ZQJSK3TQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hzjVmYJbXH9yCxL2JUotjRKfWtgdUjSQKmPRByQAMjBkHqLRK): protocol: expected piece hash; context canceled; EOF
Aug 8 06:32:40.249100268 (14h5m53.836156509s) [noise=true , address=64.49.13.154:4928 , piece_id=QDSHIQMF6WDJQZZKWQZZP4DVKFZUN2JAUVHRJOIZZ4SMNXQLVSZQ, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:126yMXrtihbzi2iQB399nmL6ARMD7NJp6Bg1YYhhVPJ7kE1DQSD): protocol: read tcp 5.161.89.252:59898->64.49.13.154:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249154668 (14h5m53.83621091s) [noise=true , address=64.49.13.27:4963 , piece_id=CEB52PLVGIB2RIXEBBSUJAQQUPGC4U2JVFQ3J7HW3LMRIFWXCAYA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n6MUXcUQ7sT61AwvkhYRq4wniRMQDWMfFk1tENVSnUiDx38rq): protocol: read tcp 5.161.89.252:44876->64.49.13.27:4963: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249199352 (14h5m53.836255594s) [noise=true , address=64.49.13.137:4504 , piece_id=UXR3Z5Q6AJWVIA27IZHNPMRTTKLC4ENHVUQHVANK5L3AXOYOOWIA, seg_pos={0 0}, upload=14568]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:11LnrSYf8ZaxUgsdjt6jHzNgHjkq1bFCnDEFd14JzhParNTesZ): protocol: read tcp 5.161.89.252:19204->64.49.13.137:4504: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Aug 8 06:32:40.249210163 (14h5m53.836266404s) [seg_pos={0 0}, upload=14568]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil>
Aug 8 06:32:40.249282678 (14h5m53.836338919s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Aug 8 06:32:40.249287737 (14h5m53.836343968s) [upload=14568]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Aug 8 06:32:40.302341870 (14h5m53.889398111s) [upload=14568]: Done uploading segments...
Aug 8 06:32:40.302361056 (14h5m53.889417297s) [upload=14568]: Upload finished. err: <nil>
Aug 8 06:33:19.671995961 (14h6m33.259052203s) [upload=14579]: Starting upload
Aug 8 06:33:19.672056584 (14h6m33.259112826s) [upload=14579]: Done starting upload
Aug 8 06:33:19.672119763 (14h6m33.259175994s) [upload=14579]: Uploading segments...
Aug 8 06:33:19.672884900 (14h6m33.259941132s) [upload=14579]: Got next segment. Inline: false
Aug 8 06:33:19.672932620 (14h6m33.259988862s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Aug 8 06:33:19.672937719 (14h6m33.259993961s) [upload=14579]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Aug 8 06:33:19.675836631 (14h6m33.262892862s) [upload=14579]: Next segment err: metaclient: Too Many Requests
Aug 8 06:33:19.675851759 (14h6m33.262908s) [upload=14579]: Done uploading segments...
Aug 8 06:33:19.675865856 (14h6m33.262922097s) [upload=14579]: Upload finished. err: metaclient: Too Many Requests
Duration: 1.051729135s
core
uplink
us-select-1
us1-placement-20
metainfo-projects
|