Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 109921 reports (Page 90 of 2199)
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.............................X..............................]
duration: 
492.129541ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:30:58 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 492.145452ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
24 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXX....................................]
duration: 
489.931816ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:30:58 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 489.947666ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [XXXXXXXXXXXXXXXXXXXXXXXXX...................................]
duration: 
487.359188ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:30:31 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 487.379476ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [..............................X.............................]
duration: 
406.854956ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:30:23 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 406.871857ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [.........XXXXXXXXXXXXXXXXXXXX.X.............................]
duration: 
667.273653ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:30:10 UTC) 
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 667.354232ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [..........XXXXXXXXXXXXXXXXXXXX.X............................]
duration: 
603.253748ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:29:51 UTC) 
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 603.277901ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [...............................X............................]
duration: 
478.57582ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:29:47 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 478.595126ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [.XXXXXXXXXXXXXXXXXXXXXXXXX..................................]
duration: 
490.124036ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:29:28 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 490.146788ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [...........XXXXXXXXXXXXXXXXXXXX.X...........................]
duration: 
617.298471ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:29:26 UTC) 
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 617.32446ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [................................X...........................]
duration: 
540.019332ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:28:54 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 540.04003ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [............XXXXXXXXXXXXXXXXXXXX.X..........................]
duration: 
648.574299ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:28:50 UTC) 
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 648.597001ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [..XXXXXXXXXXXXXXXXXXXXXXXXX.................................]
duration: 
524.272985ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:28:46 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 524.428627ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [...XXXXXXXXXXXXXXXXXXXXXXXXX................................]
duration: 
835.958649ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:28:30 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 835.983657ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [.............XXXXXXXXXXXXXXXXXXXX.X.........................]
duration: 
602.003392ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:28:07 UTC) 
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 602.032025ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [....XXXXXXXXXXXXXXXXXXXXXXXXX...............................]
duration: 
666.677478ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:27:56 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 666.704046ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.................................X..........................]
duration: 
424.308078ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:27:48 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 424.331892ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [..................................X.........................]
duration: 
406.618193ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:27:30 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 406.643992ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [.....XXXXXXXXXXXXXXXXXXXXXXXXX..............................]
duration: 
460.711109ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:27:24 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 460.732538ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [...................................X........................]
duration: 
481.053341ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:27:10 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 481.080893ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [..............XXXXXXXXXXXXXXXXXXXX.X........................]
duration: 
753.427338ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:27:03 UTC) 
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 753.460401ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [....................................X.......................]
duration: 
466.409286ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:26:26 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 466.427369ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [......XXXXXXXXXXXXXXXXXXXXXXXXX.............................]
duration: 
522.205205ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:26:17 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 522.220283ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [...............XXXXXXXXXXXXXXXXXXXX.X.......................]
duration: 
664.017407ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:26:05 UTC) 
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 664.059145ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.....................................X......................]
duration: 
496.758239ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:25:40 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 496.782154ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [................XXXXXXXXXXXXXXXXXXXX.X......................]
duration: 
665.51205ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:25:33 UTC) 
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 665.536506ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [.......XXXXXXXXXXXXXXXXXXXXXXXXX............................]
duration: 
517.003619ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:25:15 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 517.039355ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [........XXXXXXXXXXXXXXXXXXXXXXXXX...........................]
duration: 
495.857306ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:24:53 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 495.873024ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [.................XXXXXXXXXXXXXXXXXXXX.X.....................]
duration: 
770.297513ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:24:47 UTC) 
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 770.313161ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [......................................X.....................]
duration: 
489.298665ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:24:27 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 489.319263ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [..................XXXXXXXXXXXXXXXXXXXX.X....................]
duration: 
604.634798ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:24:18 UTC) 
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 604.662939ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.......................................X....................]
duration: 
450.27179ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:23:42 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 450.290144ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [.........XXXXXXXXXXXXXXXXXXXXXXXXX..........................]
duration: 
515.418848ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:23:41 UTC) 
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 515.4421ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [...................XXXXXXXXXXXXXXXXXXXX.X...................]
duration: 
570.341475ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:23:22 UTC) 
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 570.359167ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [..........XXXXXXXXXXXXXXXXXXXXXXXXX.........................]
duration: 
589.806477ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:22:46 UTC) 
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 589.823419ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [........................................X...................]
duration: 
363.295815ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:22:33 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 363.314089ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [....................XXXXXXXXXXXXXXXXXXXX.X..................]
duration: 
664.678124ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:22:08 UTC) 
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 664.699674ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [...........XXXXXXXXXXXXXXXXXXXXXXXXX........................]
duration: 
547.911728ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:21:39 UTC) 
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 547.927868ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.........................................X..................]
duration: 
450.467535ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:21:27 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 450.489576ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [.....................XXXXXXXXXXXXXXXXXXXX.X.................]
duration: 
623.05149ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:21:25 UTC) 
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 623.08285ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [............XXXXXXXXXXXXXXXXXXXXXXXXX.......................]
duration: 
515.92914ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:21:23 UTC) 
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 515.951432ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [.............XXXXXXXXXXXXXXXXXXXXXXXXX......................]
duration: 
525.365329ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:20:48 UTC) 
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 525.38775ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [......................XXXXXXXXXXXXXXXXXXXX.X................]
duration: 
581.250592ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:20:28 UTC) 
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 581.267343ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [..............XXXXXXXXXXXXXXXXXXXXXXXXX.....................]
duration: 
466.998707ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:20:19 UTC) 
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 467.044392ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration: 
498.781027ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:20:14 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 498.803489ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [...............XXXXXXXXXXXXXXXXXXXXXXXXX....................]
duration: 
487.680225ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:19:50 UTC) 
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 487.698709ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [................XXXXXXXXXXXXXXXXXXXXXXXXX...................]
duration: 
506.871556ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:19:31 UTC) 
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 506.906553ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [...........................................X................]
duration: 
459.849186ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:19:19 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 459.865767ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
21 out of 60 is failed (threshold=3) [.......................XXXXXXXXXXXXXXXXXXXX.X...............]
duration: 
617.180297ms
trace: 
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:19:16 UTC) 
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 617.199362ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
25 out of 60 is failed (threshold=6) [.................XXXXXXXXXXXXXXXXXXXXXXXXX..................]
duration: 
504.10095ms
trace: 
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:18:56 UTC) 
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 504.119294ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [............................................X...............]
duration: 
470.657486ms
trace: 
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:18:09 UTC) 
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590     , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877     , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841     , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618   , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438   , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337   , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447   , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518     , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802   , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907   , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015   , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977   , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663     , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367    , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555     , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762   , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 470.669519ms core uplink us-select-1