Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
21 out of 60 is failed (threshold=3) [.....................................XXXXXXXXXXXXXXXXXXXX.X.] duration:
983.48601ms trace:
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:07:35 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 983.504255ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
25 out of 60 is failed (threshold=6) [................................XXXXXXXXXXXXXXXXXXXXXXXXX...] duration:
931.23361ms trace:
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:07:33 UTC)
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 931.249529ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
348.777729ms trace:
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:07:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907 , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977 , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590 , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438 , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802 , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367 , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447 , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841 , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762 , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337 , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518 , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015 , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663 , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555 , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618 , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877 , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590 , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877 , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841 , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618 , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438 , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337 , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447 , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518 , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802 , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907 , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015 , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977 , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663 , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367 , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555 , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762 , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 348.796794ms
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
25 out of 60 is failed (threshold=6) [.................................XXXXXXXXXXXXXXXXXXXXXXXXX..] duration:
1.171586466s trace:
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:06:40 UTC)
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 1.171628436s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
21 out of 60 is failed (threshold=3) [......................................XXXXXXXXXXXXXXXXXXXX.X] duration:
3.429689498s trace:
https://jaeger.dev.storj.io/trace/47dedb48446d33c1
FAIL (2025-12-15 12:06:39 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881621B7F361BA2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.429708754s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.018615481s trace:
https://jaeger.dev.storj.io/trace/335d64cce74e3214
FAIL (2025-12-15 12:06:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Dec 15 12:05:56.749470185 (236h36m44.447038532s) [upload=207961]: Starting upload
Dec 15 12:05:56.749544054 (236h36m44.447112389s) [upload=207961]: Done starting upload
Dec 15 12:05:56.749767430 (236h36m44.447335766s) [upload=207961]: Uploading segments...
Dec 15 12:05:56.749958127 (236h36m44.447526463s) [upload=207961]: Got next segment. Inline: false
Dec 15 12:05:56.750013350 (236h36m44.447581687s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Dec 15 12:05:56.750019502 (236h36m44.447587848s) [upload=207961]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Dec 15 12:05:56.797666136 (236h36m44.495234473s) [seg_pos={0 0}, upload=207961]: Begin upload segment...
Dec 15 12:05:56.797704828 (236h36m44.495273164s) [seg_pos={0 0}, upload=207961]: Done begin upload segment.
Dec 15 12:05:56.797745896 (236h36m44.495314241s) [noise=true , address=145.79.227.34:4907 , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.797796150 (236h36m44.495364486s) [noise=true , address=145.79.227.38:4977 , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798447166 (236h36m44.496015501s) [noise=true , address=77.90.31.12:4590 , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.798995690 (236h36m44.496564026s) [noise=true , address=145.79.227.20:4438 , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799074017 (236h36m44.496642362s) [noise=true , address=145.79.227.13:4802 , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799436463 (236h36m44.49700481s) [noise=true , address=145.79.227.9:4367 , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799746313 (236h36m44.497314648s) [noise=true , address=145.79.227.52:4447 , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.799842131 (236h36m44.497410467s) [noise=true , address=77.90.31.19:4841 , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800323250 (236h36m44.497891596s) [noise=true , address=145.79.227.30:4762 , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800431692 (236h36m44.498000029s) [noise=true , address=145.79.227.49:4337 , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.800806453 (236h36m44.498374799s) [noise=true , address=77.90.31.10:4518 , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801016085 (236h36m44.49858442s) [noise=true , address=145.79.227.39:4015 , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801225366 (236h36m44.498793701s) [noise=true , address=77.90.31.14:4663 , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.801636644 (236h36m44.49920498s) [noise=true , address=77.90.31.11:4555 , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.802364023 (236h36m44.499932359s) [noise=true , address=145.79.227.25:4618 , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.803096520 (236h36m44.500664866s) [noise=true , address=77.90.31.21:4877 , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Uploading piece...
Dec 15 12:05:56.854745603 (236h36m44.552313949s) [noise=true , address=77.90.31.12:4590 , piece_id=4U36PUF54TCKASBIBIHXYKDGPAZYAGUN2CVS2VUAGYYFXYVXQIIQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856474272 (236h36m44.554042609s) [noise=true , address=77.90.31.21:4877 , piece_id=G73WQWYWMCEPCKE5654QCHGZ25TCUGTPG36PM4DCS2NGRD7GNDWQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.856729499 (236h36m44.554297836s) [noise=true , address=77.90.31.19:4841 , piece_id=7537WGH3U5LJRDCTFGBI4NIKVRD4VEPQGSFGYTGXU2U3G6QH4KEQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000
Dec 15 12:05:56.914070765 (236h36m44.611639101s) [upload=207961]: Next returned nil segment
Dec 15 12:05:56.914084190 (236h36m44.611652526s) [upload=207961]: Waiting for error group managing segments...
Dec 15 12:05:56.928069265 (236h36m44.625637601s) [noise=true , address=145.79.227.25:4618 , piece_id=UAB6NPZOX4BKICPAOO2XOE5QASMI5CC54XNGJXT73J6S3QBGZT5Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929331853 (236h36m44.626900189s) [noise=true , address=145.79.227.20:4438 , piece_id=4L6DGNWJR3Z57EW3OMNTNXHFWSNSMOVTX2YVFMO3WTAHKT6RCOQA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.929375425 (236h36m44.626943761s) [noise=true , address=145.79.227.49:4337 , piece_id=KOW5EWZU6U5P77WKUHAKDA557YCNVO2OMZBPC3LE4GTDY5BDHAMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930011252 (236h36m44.627579588s) [noise=true , address=145.79.227.52:4447 , piece_id=HG7MJFAZEKTYP57A52KHYS55G5U4AWKHC2JJBS4PXCPSRXORW57Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930187632 (236h36m44.627755978s) [noise=true , address=77.90.31.10:4518 , piece_id=FTVDWDRNK6GBJ2TDYCED4JOHMP64SZQNS4SHBYGPHBLYVFLNDCMQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930582028 (236h36m44.628150365s) [noise=true , address=145.79.227.13:4802 , piece_id=YAWZYSRM44KOTQO74MJNBFMB3LOODWIQICF46KVQTBV3JIX7OQVQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.930801609 (236h36m44.628369945s) [noise=true , address=145.79.227.34:4907 , piece_id=JUIWXGXIAVQF4QWPBUGG47GI3WYXNUE6DMTOJGJTWP3EO2RSUMZA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931344463 (236h36m44.628912799s) [noise=true , address=145.79.227.39:4015 , piece_id=I22GWL3PB2TIRPOMK72LP2DKVIPCZ2MFLNN4MD3EHMCZ6X4AQ7QA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931489714 (236h36m44.629058051s) [noise=true , address=145.79.227.38:4977 , piece_id=GIGVPLJDSMQT6UMPLEOMHFUNXI5DHSSZJVU3P6QPHU6GXZ5633LQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.931962767 (236h36m44.629531104s) [noise=true , address=77.90.31.14:4663 , piece_id=42DU462B3UGWGR3QBBSSCAESUEL5SODQKCL52YYSVPCUGNLVW56A, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:56.932184141 (236h36m44.629752478s) [noise=true , address=145.79.227.9:4367 , piece_id=X3ADNSXPCUVPAKKD6IZQGPYYZQY5VSTHKWQQ6YHRGZ67HZ3U7BTA, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:05:57.129527226 (236h36m44.827095563s) [noise=true , address=77.90.31.11:4555 , piece_id=VMOH4OXBNJPUIJSC3S4X2DWK73AXF567RTSMJFFDOTMUEBSEYJAQ, seg_pos={0 0}, upload=207961]: Done uploading piece. err: <nil>
Dec 15 12:06:26.704458766 (236h37m14.402027102s) [noise=true , address=145.79.227.30:4762 , piece_id=TEBIX4WUG2SWR5KKA7Z6MMYPHIJPSHBAKQKT3XMAZQBS7YLW632Q, seg_pos={0 0}, upload=207961]: Done uploading piece. err: ecclient: upload failed (node:1No1jHEK6YFSye92GGBri48Yv2DYUiESY6Gs3Z9PENjmBynS9b, address:145.79.227.30:4762): protocol: context deadline exceeded; context deadline exceeded
Dec 15 12:06:26.704503140 (236h37m14.402071475s) [seg_pos={0 0}, upload=207961]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Dec 15 12:06:26.719801077 (236h37m14.417369413s) [upload=207961]: Done uploading segments...
Dec 15 12:06:26.719830712 (236h37m14.417399049s) [upload=207961]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 30.018640979s
core
uplink
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
20 out of 60 is failed (threshold=3) [.......................................XXXXXXXXXXXXXXXXXXXX.] duration:
5.699050907s trace:
https://jaeger.dev.storj.io/trace/29b9e72573ba3e3d
FAIL (2025-12-15 12:06:15 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 18816207B5FA15AE, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 5.699075432s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
25 out of 60 is failed (threshold=6) [..................................XXXXXXXXXXXXXXXXXXXXXXXXX.] duration:
617.175927ms trace:
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:05:45 UTC)
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 617.206596ms
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
20 out of 60 is failed (threshold=3) [........................................XXXXXXXXXXXXXXXXXXXX] duration:
4.822528856s trace:
https://jaeger.dev.storj.io/trace/29b9e72573ba3e3d
FAIL (2025-12-15 12:05:14 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 18816207B5FA15AE, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 4.822547913s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
25 out of 60 is failed (threshold=6) [...................................XXXXXXXXXXXXXXXXXXXXXXXXX] duration:
5.000521781s trace:
https://jaeger.dev.storj.io/trace/140cd29facd4e598
FAIL (2025-12-15 12:04:56 UTC)
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 5.000534174s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
19 out of 60 is failed (threshold=3) [.........................................XXXXXXXXXXXXXXXXXXX] duration:
1.839329765s trace:
https://jaeger.dev.storj.io/trace/2caf4ea2a305855e
FAIL (2025-12-15 12:04:07 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161F81F144A74, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 1.839345985s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
24 out of 60 is failed (threshold=6) [....................................XXXXXXXXXXXXXXXXXXXXXXXX] duration:
4.003186817s trace:
https://jaeger.dev.storj.io/trace/29f2dd2370f30917
FAIL (2025-12-15 12:03:47 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161F3999FA551, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 4.003202045s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
23 out of 60 is failed (threshold=6) [.....................................XXXXXXXXXXXXXXXXXXXXXXX] duration:
2.95675903s trace:
https://jaeger.dev.storj.io/trace/732b6cfdaa0f73ce
FAIL (2025-12-15 12:03:21 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161ED5F37D456, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.956782073s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
18 out of 60 is failed (threshold=3) [..........................................XXXXXXXXXXXXXXXXXX] duration:
2.159611267s trace:
https://jaeger.dev.storj.io/trace/636796ba2ef97454
FAIL (2025-12-15 12:03:20 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161ED5152D8D5, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.159630673s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
17 out of 60 is failed (threshold=3) [...........................................XXXXXXXXXXXXXXXXX] duration:
4.802006138s trace:
https://jaeger.dev.storj.io/trace/3604d064eb1fdb9a
FAIL (2025-12-15 12:02:49 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161E5FC9B53CD, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 4.802022849s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
22 out of 60 is failed (threshold=6) [......................................XXXXXXXXXXXXXXXXXXXXXX] duration:
4.389898741s trace:
https://jaeger.dev.storj.io/trace/4077130f90cfc5c6
FAIL (2025-12-15 12:02:25 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161E05C7B48E2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 4.389916905s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
21 out of 60 is failed (threshold=6) [.......................................XXXXXXXXXXXXXXXXXXXXX] duration:
3.82015209s trace:
https://jaeger.dev.storj.io/trace/68d9ae0d7f0a3443
FAIL (2025-12-15 12:01:56 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161D9C4B736B2, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.820169891s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
16 out of 60 is failed (threshold=3) [............................................XXXXXXXXXXXXXXXX] duration:
2.625475563s trace:
https://jaeger.dev.storj.io/trace/550ed53e0037ebff
FAIL (2025-12-15 12:01:40 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161D5EE6A8E6E, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.625490261s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
20 out of 60 is failed (threshold=6) [........................................XXXXXXXXXXXXXXXXXXXX] duration:
3.70614338s trace:
https://jaeger.dev.storj.io/trace/7d6bdd4f29e01159
FAIL (2025-12-15 12:01:34 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161D48EF8ADE9, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.706161805s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
15 out of 60 is failed (threshold=3) [.............................................XXXXXXXXXXXXXXX] duration:
2.638764415s trace:
https://jaeger.dev.storj.io/trace/ced699de8c9485
FAIL (2025-12-15 12:00:52 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161CAC9869C97, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.638780796s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
19 out of 60 is failed (threshold=6) [.........................................XXXXXXXXXXXXXXXXXXX] duration:
5.000426184s trace:
https://jaeger.dev.storj.io/trace/5f949a2555e7d625
FAIL (2025-12-15 12:00:28 UTC)
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 5.000442845s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
14 out of 60 is failed (threshold=3) [..............................................XXXXXXXXXXXXXX] duration:
2.374433172s trace:
https://jaeger.dev.storj.io/trace/373a5b8a8e88d55c
FAIL (2025-12-15 12:00:22 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161C3B98CDDAC, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.374450644s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [...............................................XXXXXXXXXXXXX] duration:
3.13354771s trace:
https://jaeger.dev.storj.io/trace/4d61158984b70883
FAIL (2025-12-15 12:00:03 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161BF702BFEF8, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.133570282s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
18 out of 60 is failed (threshold=6) [..........................................XXXXXXXXXXXXXXXXXX] duration:
3.019959615s trace:
https://jaeger.dev.storj.io/trace/224d2ee5ed8eb005
FAIL (2025-12-15 11:59:33 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161B85BA131C8, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.019977909s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
17 out of 60 is failed (threshold=6) [...........................................XXXXXXXXXXXXXXXXX] duration:
3.29630218s trace:
https://jaeger.dev.storj.io/trace/5179beadf6ed38e7
FAIL (2025-12-15 11:59:07 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161B2643524F3, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.296313632s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [................................................XXXXXXXXXXXX] duration:
2.959041196s trace:
https://jaeger.dev.storj.io/trace/5da42432f3a82ba2
FAIL (2025-12-15 11:59:00 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161B09DA4094B, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.959060973s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
16 out of 60 is failed (threshold=6) [............................................XXXXXXXXXXXXXXXX] duration:
4.355120675s trace:
https://jaeger.dev.storj.io/trace/248dd45b8013561e
FAIL (2025-12-15 11:58:32 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161AA434E669E, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 4.355138288s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
15 out of 60 is failed (threshold=6) [.............................................XXXXXXXXXXXXXXX] duration:
1.77629941s trace:
https://jaeger.dev.storj.io/trace/5f4c312cef0081ae
FAIL (2025-12-15 11:58:00 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161A2B216F8BD, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 1.776316832s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [.................................................XXXXXXXXXXX] duration:
4.607820627s trace:
https://jaeger.dev.storj.io/trace/3512805241d79516
FAIL (2025-12-15 11:57:50 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161A05D2E2CBF, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 4.607842367s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
14 out of 60 is failed (threshold=6) [..............................................XXXXXXXXXXXXXX] duration:
4.89366436s trace:
https://jaeger.dev.storj.io/trace/2bfc754cdc4b5f07
FAIL (2025-12-15 11:57:34 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881619C8BA74F0C, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 11))
Duration: 4.893687704s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
13 out of 60 is failed (threshold=6) [...............................................XXXXXXXXXXXXX] duration:
1.04318517s trace:
https://jaeger.dev.storj.io/trace/545e2278a9c2c154
FAIL (2025-12-15 11:57:02 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881619537E6A7F5, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 1.043208144s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
10 out of 60 is failed (threshold=3) [..................................................XXXXXXXXXX] duration:
3.584848052s trace:
https://jaeger.dev.storj.io/trace/5b3305ef1d6b362
FAIL (2025-12-15 11:56:51 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881619296E725D3, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.58490536s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
12 out of 60 is failed (threshold=6) [................................................XXXXXXXXXXXX] duration:
3.590706917s trace:
https://jaeger.dev.storj.io/trace/771be5f8014a1711
FAIL (2025-12-15 11:56:32 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881618E4EA7C78D, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 11))
Duration: 3.590725462s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [...................................................XXXXXXXXX] duration:
3.811810815s trace:
https://jaeger.dev.storj.io/trace/4801ea9a872881f8
FAIL (2025-12-15 11:55:50 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161845C99966B, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.811843606s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
11 out of 60 is failed (threshold=6) [.................................................XXXXXXXXXXX] duration:
2.913714223s trace:
https://jaeger.dev.storj.io/trace/7806d48739005e60
FAIL (2025-12-15 11:55:46 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881618397B3AB5B, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.913740292s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
10 out of 60 is failed (threshold=6) [..................................................XXXXXXXXXX] duration:
5.000399777s trace:
https://jaeger.dev.storj.io/trace/7d8da6d588930dc6
FAIL (2025-12-15 11:55:21 UTC)
operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 5.000412971s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
9 out of 60 is failed (threshold=6) [...................................................XXXXXXXXX] duration:
2.048662175s trace:
https://jaeger.dev.storj.io/trace/78ce2af96503f332
FAIL (2025-12-15 11:54:50 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 18816176761F4EC3, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.048681813s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [....................................................XXXXXXXX] duration:
1.769096163s trace:
https://jaeger.dev.storj.io/trace/6ffb98e593d0f15d
FAIL (2025-12-15 11:54:46 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 18816175817CA63A, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 1.76911537s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
8 out of 60 is failed (threshold=6) [....................................................XXXXXXXX] duration:
1.53831085s trace:
https://jaeger.dev.storj.io/trace/2f93d69f34e7862
FAIL (2025-12-15 11:54:01 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881616B02202B0E, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 1.538328152s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [.....................................................XXXXXXX] duration:
3.611127053s trace:
https://jaeger.dev.storj.io/trace/12bd13ad6e6477dc
FAIL (2025-12-15 11:54:01 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881616B0EEB399D, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.611149525s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
7 out of 60 is failed (threshold=6) [.....................................................XXXXXXX] duration:
4.911597382s trace:
https://jaeger.dev.storj.io/trace/374be87641d6b45
FAIL (2025-12-15 11:53:36 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 18816165420AA131, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 4.911614723s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [......................................................XXXXXX] duration:
3.828418398s trace:
https://jaeger.dev.storj.io/trace/7d89a4a0d6c7f9b9
FAIL (2025-12-15 11:53:21 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 18816161B6AADBEB, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 3.828436482s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
6 out of 60 is failed (threshold=6) [......................................................XXXXXX] duration:
1.550973155s trace:
https://jaeger.dev.storj.io/trace/16097658e208f331
FAIL (2025-12-15 11:52:44 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161592F0F67F3, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 1.550994635s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.......................................................XXXXX] duration:
5.003805978s trace:
https://jaeger.dev.storj.io/trace/6961f2ca204b50d2
FAIL (2025-12-15 11:52:19 UTC)
orbiter warning: operation error S3: PutObject, request canceled, context deadline exceeded
Duration: 5.003823631s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.......................................................XXXXX] duration:
2.962345887s trace:
https://jaeger.dev.storj.io/trace/855abf0231a3cab
FAIL (2025-12-15 11:52:12 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 18816151C652DD34, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.962362408s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [........................................................XXXX] duration:
2.585031411s trace:
https://jaeger.dev.storj.io/trace/5aa622740348f5d2
FAIL (2025-12-15 11:51:52 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881614D011DA458, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.585057129s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
4 out of 60 is failed (threshold=6) [........................................................XXXX] duration:
2.439735367s trace:
https://jaeger.dev.storj.io/trace/2bce81d364ec261
FAIL (2025-12-15 11:51:15 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881614477AC6936, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.439756908s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.........................................................XXX] duration:
4.818175003s trace:
https://jaeger.dev.storj.io/trace/5d4933dd920a45b0
FAIL (2025-12-15 11:51:09 UTC)
operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 188161431262E411, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 4.818191403s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
2 out of 60 is failed (threshold=3) [..........................................................XX] duration:
4.427746876s trace:
https://jaeger.dev.storj.io/trace/1ed71865e0add4c8
FAIL (2025-12-15 11:50:39 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881613C119FF667, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 4.427765762s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.........................................................XXX] duration:
2.834834547s trace:
https://jaeger.dev.storj.io/trace/f1342226597d54e
FAIL (2025-12-15 11:50:38 UTC)
orbiter warning: operation error S3: PutObject, exceeded maximum number of attempts, 3, https response error StatusCode: 500, RequestID: 1881613BE864316A, HostID: , api error InternalError: We encountered an internal error, please try again.: cause(uplink: failed to upload enough pieces (needed at least 14 but got 10))
Duration: 2.834880692s
core
uplink
us-select-1
us1-placement-20
|