Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 365.637144ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:19:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 365.669463ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 687.544154ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:19:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 687.566135ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........XX...............................................] duration: 813.124862ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:18:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 813.163044ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 776.119507ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:18:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 776.135909ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 319.045143ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:18:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 319.057095ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [............XX..............................................] duration: 729.002908ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:18:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 729.023305ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 293.094328ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:17:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 293.110679ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 595.621707ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:17:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 595.661462ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.............XX.............................................] duration: 479.385782ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:17:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 479.466383ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 316.772318ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:16:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 316.790562ms core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 359.84235ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:16:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 359.86361ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 619.004933ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:16:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 619.029438ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 551.354216ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:15:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 551.37227ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............XX............................................] duration: 487.82685ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:15:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 487.840023ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 375.809015ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:15:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 375.825856ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............XX...........................................] duration: 499.449722ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:15:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 499.471803ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 620.379242ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:15:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 620.404982ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 321.735529ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:14:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 321.751709ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................XX..........................................] duration: 484.420116ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:14:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 484.444361ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 591.020552ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:14:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 591.035059ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 606.546472ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:13:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 606.568142ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 377.257088ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:13:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 377.321249ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.................XX.........................................] duration: 486.035603ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:13:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 486.052164ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 355.085386ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:13:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 355.101938ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..................XX........................................] duration: 580.797063ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:12:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 580.818493ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 623.514508ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:12:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 623.548731ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 369.787486ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:12:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 369.814085ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...................XX.......................................] duration: 665.859385ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:12:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 665.874915ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 614.787951ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:12:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 614.808309ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 336.185349ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:11:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 336.19629ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....................XX......................................] duration: 662.40718ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:11:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 662.424924ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....................XX.....................................] duration: 466.201638ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:10:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 466.238978ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 619.10288ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:10:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 619.12435ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 373.274908ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:10:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 373.432443ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......................XX....................................] duration: 531.64139ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:10:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 531.661276ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 622.968789ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:10:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 622.980902ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......................XX...................................] duration: 503.919553ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:10:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 503.935582ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 625.953112ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:09:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 625.967558ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 341.509077ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:09:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 341.526191ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 576.734653ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:09:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 576.752014ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 310.607677ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:08:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 310.626453ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........................XX..................................] duration: 493.864143ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:08:52 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 493.892125ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 367.592263ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:08:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 367.611489ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........................XX.................................] duration: 501.554024ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:08:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 501.576666ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 593.554097ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:08:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 593.565939ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................XX................................] duration: 549.236044ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:07:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 549.251824ms core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 367.168684ms trace: https://jaeger.dev.storj.io/trace/3bb5e6bb5aa7ad6b FAIL (2026-01-26 19:07:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Jan 26 18:43:22.244729209 (266h47m0.42717419s) [upload=234289]: Starting upload Jan 26 18:43:22.244783651 (266h47m0.427228631s) [upload=234289]: Done starting upload Jan 26 18:43:22.244862287 (266h47m0.427307267s) [upload=234289]: Uploading segments... Jan 26 18:43:22.245010364 (266h47m0.427455334s) [upload=234289]: Got next segment. Inline: false Jan 26 18:43:22.245044468 (266h47m0.427489448s) [upload=234289]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jan 26 18:43:22.245048175 (266h47m0.427493155s) [upload=234289]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jan 26 18:43:22.266231092 (266h47m0.448676072s) [seg_pos={0 0}, upload=234289]: Begin upload segment... Jan 26 18:43:22.266262791 (266h47m0.448707761s) [seg_pos={0 0}, upload=234289]: Done begin upload segment. Jan 26 18:43:22.266285364 (266h47m0.448730334s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266372465 (266h47m0.448817446s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.266739602 (266h47m0.449184583s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267056043 (266h47m0.449501013s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267372755 (266h47m0.449817725s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267627821 (266h47m0.450072801s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267660202 (266h47m0.450105171s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.267886355 (266h47m0.450331325s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268219706 (266h47m0.450664677s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268756349 (266h47m0.451201319s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268762169 (266h47m0.451207149s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.268800502 (266h47m0.451245472s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269264038 (266h47m0.451709008s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269456296 (266h47m0.451901276s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.269827900 (266h47m0.45227287s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.270230122 (266h47m0.452675091s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Uploading piece... Jan 26 18:43:22.346167333 (266h47m0.528612314s) [upload=234289]: Next returned nil segment Jan 26 18:43:22.346177152 (266h47m0.528622132s) [upload=234289]: Waiting for error group managing segments... Jan 26 18:43:22.355942092 (266h47m0.538387072s) [noise=true , address=145.79.227.52:4446 , piece_id=SWREVABTLINLIKWZ5T4K4F6TAC5VF4PWHPPTIT2HVMEBFHRSNKEA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.356544547 (266h47m0.538989517s) [noise=true , address=145.79.227.51:4411 , piece_id=QX42UUIQRKDLVW2P4MDHGVM7YBDK4TB2LENZBVDLS4RMUQLPLHNQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357475265 (266h47m0.539920235s) [noise=true , address=145.79.227.48:4301 , piece_id=JYZZKQCYSWGQD3JTFIXNQ2UNLU4H2K7D2EAEB7EILRUHHWZG6X5A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357741102 (266h47m0.540186082s) [noise=true , address=145.79.227.47:4265 , piece_id=RQLMIDMVU45GIR2VOBZQRVLZEEKI4TEU4EJVTLLNLKDHAS3GSGTQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.357892725 (266h47m0.540337705s) [noise=true , address=145.79.227.24:4583 , piece_id=7ITWHDHIII3LFUA2AD3UQBXFXP43CFDIK4NKHYDD2BTGJII2WRBA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358205700 (266h47m0.54065068s) [noise=true , address=145.79.227.39:4013 , piece_id=IDWYWDFFNKZN5EY2THLNG57FF3JQNQVFVU6RF4MACLIEDF7ZV37A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358277304 (266h47m0.540722284s) [noise=true , address=145.79.227.46:4601 , piece_id=AWDYHRIULVSHJY4MMGUTV54D7WBTSRWTXUWX35HIYDNB63I4QPFQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.358633879 (266h47m0.541078859s) [noise=true , address=145.79.227.13:4803 , piece_id=TVP3SVAKCCCFAOOVG3DAPLEAAL3IFP76TCQ4LN43I2TQNDZT7DGA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359212861 (266h47m0.541657832s) [noise=true , address=145.79.227.33:4871 , piece_id=2VUU7K3BCWPKKHHU7CAXEBRITS6ED6WDFPU3ODBG2YPDTCZKWD3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.359410610 (266h47m0.54185559s) [noise=true , address=145.79.227.22:4511 , piece_id=OGUKV6FBZNOYNTUO3A7M3Z532KQZH2K5AQIDPUKD3RYDPXFHBAEQ, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:22.374088006 (266h47m0.556532976s) [noise=true , address=145.79.227.37:4565 , piece_id=JQVIFKPIBMXFTYYTHF3Q7DKEIFSMKV6FBXFOIZ4HJEDCT3S7YGJA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: <nil> Jan 26 18:43:52.233014687 (266h47m30.415459677s) [noise=true , address=45.91.116.9:4261 , piece_id=BWWCW3CPBTPWOF4K2NUOXEZRMHZGILDOT335RY5TSZUJSETSF6BA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:20506->45.91.116.9:4261: use of closed network connection Jan 26 18:43:52.233783362 (266h47m30.416228333s) [noise=true , address=45.91.116.7:4010 , piece_id=3RADWMUAMQOTQQF3YLXSJJOWSQRZEWVZJQXPHMADDWYQIKJX6UCA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233815402 (266h47m30.416260362s) [noise=true , address=45.91.116.14:4191 , piece_id=HAYM5NIRM2ELGZ7FO7L2EY3HMAAED7JWEC4V7SD3BDHEADELNWUA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: read tcp 5.161.89.252:12188->45.91.116.14:4191: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233921791 (266h47m30.416366771s) [noise=true , address=45.91.116.6:4976 , piece_id=6RKPBVLLDTIEYOBANAB4NRDYAY353LUCVAOAS2523RTXCVCDVO3A, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:121tumRJ6tGiip2dumJr9ZWMoU5BA2HuLEwU5GxBxzTZjMhv3Pp, address:45.91.116.6:4976): protocol: write tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; close tcp 5.161.89.252:62910->45.91.116.6:4976: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jan 26 18:43:52.233966906 (266h47m30.416411876s) [noise=true , address=45.91.116.12:4225 , piece_id=M7NFP2QJD5JTMJXROZSXKW6LAJ45WTJJ4XOJLBD4UQEDJJSEY4RA, seg_pos={0 0}, upload=234289]: Done uploading piece. err: ecclient: upload failed (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy, address:45.91.116.12:4225): protocol: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection; piecestore: piecestore close: write tcp 5.161.89.252:36958->45.91.116.12:4225: use of closed network connection Jan 26 18:43:52.233984328 (266h47m30.416429288s) [seg_pos={0 0}, upload=234289]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Jan 26 18:43:52.255878794 (266h47m30.438323804s) [upload=234289]: Done uploading segments... Jan 26 18:43:52.255942072 (266h47m30.438387072s) [upload=234289]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 367.185326ms core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 601.312529ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:07:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 601.32946ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 635.851057ms trace: https://jaeger.dev.storj.io/trace/6fe61816dac9c35f FAIL (2026-01-26 19:06:58 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 635.872528ms core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_us_qa_orbiter_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........................XX...............................] duration: 513.68504ms trace: https://jaeger.dev.storj.io/trace/460a0499eabe6dd0 FAIL (2026-01-26 19:06:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 513.704837ms core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)