Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 122745 reports (Page 86 of 2455)
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
587.158371ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:54:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 587.196243ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [.............................................XX.............]
duration:
523.693275ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:53:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 523.723533ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
631.539689ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:53:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 631.563093ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration:
319.050841ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:53:33 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 319.068424ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [..............................................XX............]
duration:
495.584298ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:53:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 495.60094ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
304.790962ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:52:47 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 304.808595ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
626.745416ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:52:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 626.76885ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
414.836845ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:52:25 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 414.863615ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
593.675405ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:52:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 593.701292ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [...............................................XX...........]
duration:
470.306225ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:51:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 470.326153ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
370.956602ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:51:41 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 370.985735ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
424.525392ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:51:24 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 424.537264ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
567.877716ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:51:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 567.893515ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [................................................XX..........]
duration:
524.713072ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:51:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 524.725746ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
370.878426ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:51:04 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 370.89701ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
553.37216ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:50:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 553.38782ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
606.52181ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:50:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 606.694422ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
388.572658ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:50:19 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 388.59569ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [.................................................XX.........]
duration:
507.080787ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:50:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 507.098429ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [..................................................XX........]
duration:
500.538415ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:49:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 500.571378ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
360.543481ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:49:31 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 360.555415ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
647.967965ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:49:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 647.994345ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [...................................................XX.......]
duration:
498.506933ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:48:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 498.523913ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
381.081354ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:48:38 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 381.102123ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........]
duration:
316.609877ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:48:23 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 316.626679ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........]
duration:
698.093805ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:48:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 698.109204ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......]
duration:
303.274349ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:47:34 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 303.291321ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [....................................................XX......]
duration:
620.085486ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:47:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 620.125519ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
334.409505ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:47:17 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 334.431406ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......]
duration:
645.354275ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:47:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 645.378209ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [.....................................................XX.....]
duration:
572.414699ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:46:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 572.434025ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
646.189364ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:46:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 646.207849ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
851.41842ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:46:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 851.436143ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
335.720583ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:46:10 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 335.74031ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [......................................................XX....]
duration:
551.70156ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:46:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 551.729091ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
363.981854ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:45:55 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 364.002893ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
582.371041ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:45:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 582.384105ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
384.814086ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:45:21 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 384.833852ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [.......................................................XX...]
duration:
517.796111ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:45:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 517.814876ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
610.379626ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:45:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 610.395725ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [........................................................XX..]
duration:
490.723453ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:44:49 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 490.878994ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
332.394293ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:44:48 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 332.415392ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
650.739423ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:44:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 650.763167ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
459.898025ms
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:44:16 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 459.920035ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [.........................................................XX.]
duration:
803.727397ms
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:44:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 803.744669ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
30.025120459s
trace:
https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b
FAIL (2026-01-26 18:43:52 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload
Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload
Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments...
Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false
Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment...
Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment.
Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece...
Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment
Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments...
Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446   , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411   , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301   , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265   , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583   , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013   , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601   , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803   , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871   , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511   , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565   , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil>
Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261     , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection
Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010     , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191    , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976     , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225    , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection
Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments...
Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 30.025133905s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
601.256494ms
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:43:51 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 601.267544ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
1m0.000815975s
trace:
https://jaeger.dev.storj.io/trace/6fe61816dac9c35f
FAIL (2026-01-26 18:43:50 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.00082944s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
2 out of 60 is failed (threshold=6) [..........................................................XX]
duration:
5.001394017s
trace:
https://jaeger.dev.storj.io/trace/460a0499eabe6dd0
FAIL (2026-01-26 18:43:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001409306s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)
antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X]
duration:
5.001778686s
trace:
https://jaeger.dev.storj.io/trace/11fb3afd94cc2a3f
FAIL (2026-01-26 18:43:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.00179691s core uplink us-select-1 us1-placement-20