Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
345.391812ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:13:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 345.417398ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
337.395745ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:12:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 337.415632ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
312.337163ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:11:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 312.358614ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
332.368406ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:11:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 332.38731ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
333.451738ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:10:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 333.465795ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
312.431889ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:10:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 312.453129ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
392.867694ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:09:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 392.886619ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
364.102444ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:08:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 364.122642ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
302.500437ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:06:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 302.516708ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
358.950141ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:06:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 358.994153ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
347.631789ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:04:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 347.660683ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
334.498185ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:04:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 334.516189ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
360.896806ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:03:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 360.917986ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
394.302204ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:03:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 394.324777ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
333.076985ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:02:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 333.103965ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
354.935514ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:01:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 354.965569ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
355.967541ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:01:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 355.985794ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
378.194243ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:01:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 378.212097ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
293.115818ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 22:00:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 293.13837ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
412.838673ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:58:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 412.85838ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
367.235533ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:58:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 367.255359ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
303.688869ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:58:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 303.707986ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
396.986962ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:56:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 397.05543ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
308.275125ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:56:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 308.292919ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
427.167414ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:55:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 427.178395ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
320.09049ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:54:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 320.113403ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
283.566929ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:53:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 283.584912ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
419.065617ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:52:20 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 419.091656ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
455.582485ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:51:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 455.603724ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
357.439131ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:51:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 357.466732ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
378.196609ms trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:50:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 378.22344ms
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
8.020488485s trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:49:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 8.020508434s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_us_qa_orbiter_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.017262529s trace:
https://jaeger.dev.storj.io/trace/3abf05abc2bcbbe9
FAIL (2026-02-01 21:49:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 9)
======LOGS======
Feb 1 21:48:33.847424731 (413h52m12.029869702s) [upload=363886]: Starting upload
Feb 1 21:48:33.847468233 (413h52m12.029913213s) [upload=363886]: Done starting upload
Feb 1 21:48:33.847541580 (413h52m12.02998655s) [upload=363886]: Uploading segments...
Feb 1 21:48:33.847687482 (413h52m12.030132463s) [upload=363886]: Got next segment. Inline: false
Feb 1 21:48:33.847720684 (413h52m12.030165664s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Feb 1 21:48:33.847724482 (413h52m12.030169452s) [upload=363886]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Feb 1 21:48:33.868626379 (413h52m12.051071359s) [seg_pos={0 0}, upload=363886]: Begin upload segment...
Feb 1 21:48:33.868665933 (413h52m12.051110913s) [seg_pos={0 0}, upload=363886]: Done begin upload segment.
Feb 1 21:48:33.868688756 (413h52m12.051133736s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.868709133 (413h52m12.051154113s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869115594 (413h52m12.051560574s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.869634212 (413h52m12.052079192s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.870315244 (413h52m12.052760214s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871120198 (413h52m12.053565188s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.871986856 (413h52m12.054431866s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.872591025 (413h52m12.055036035s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873455360 (413h52m12.05590033s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.873799974 (413h52m12.056244944s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.875187926 (413h52m12.057632906s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.876363683 (413h52m12.058808653s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.877488204 (413h52m12.059933174s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.880294336 (413h52m12.062739316s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886041933 (413h52m12.068486914s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.886737964 (413h52m12.069182944s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Uploading piece...
Feb 1 21:48:33.948117037 (413h52m12.130562017s) [upload=363886]: Next returned nil segment
Feb 1 21:48:33.948125313 (413h52m12.130570282s) [upload=363886]: Waiting for error group managing segments...
Feb 1 21:48:33.957333384 (413h52m12.139778365s) [noise=true , address=45.91.116.14:4189 , piece_id=HNMW5ERG7VVZWZLSDAKTXZUG3CUSOK34NY5QWBWKEYMOUOH7W7XA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957434884 (413h52m12.139879854s) [noise=true , address=45.91.116.13:4155 , piece_id=POJ5UE2O7CCLG4RCV6Y36S5A22NVXV3EP2MEC2G2SCVVYMW34Y7Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957763769 (413h52m12.140208749s) [noise=true , address=45.91.116.6:4989 , piece_id=7E7IYYNMBDFXZAKM76JX55WNMKNBPO3XPSVOQLA3U4HXYPNFWIYQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.957935891 (413h52m12.140380861s) [noise=true , address=77.90.31.15:4700 , piece_id=SCYFAE6BEMDU3D4RHZ2RAX5S6UJVJHNXYED2QR6SVUMEBRANSSAA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958167643 (413h52m12.140612613s) [noise=true , address=77.90.31.17:4770 , piece_id=47HLQILNFXUUTM3CVNSMJQAPYAWYSVPUB7QHHTWHVLHRV26JA5JQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958467995 (413h52m12.140912964s) [noise=true , address=77.90.31.13:4629 , piece_id=M7IHM45AK63MBSG75YFBJNGD6FBA3M5T5HTKCIOUOY4UL6JLBDRA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.958690309 (413h52m12.141135279s) [noise=true , address=77.90.31.12:4589 , piece_id=KLHFY76F5L76H75QWUFVBNJWKWSORRWRPE73O4ZXACT5JFC4FIOQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.959596913 (413h52m12.142041884s) [noise=true , address=45.91.116.10:4082 , piece_id=M4V73KECNHBIOH7XQMRU27MIEVODSWGADY2UMZO4GOI2KFYELICQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:48:33.962301265 (413h52m12.144746235s) [noise=true , address=77.90.31.18:4805 , piece_id=FZBCSVJZTSXCZOPXGOEOQVUIZTY76BSGBDO2I75MFQPWHWZAQPVA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: <nil>
Feb 1 21:49:03.830643113 (413h52m42.013088093s) [noise=true , address=145.79.227.49:4339 , piece_id=XO2YCU6ICRSF7O6G6ZPZIRVFQTGIP6H2QZUCMEBQQ6UX6VFFNB2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:124ACRNWgJYWgCVsDkXchmT47DiUcgvtZcdGo2MzyVqZkPjyz1R, address:145.79.227.49:4339): protocol: rpc: dial tcp 145.79.227.49:4339: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830689109 (413h52m42.013134089s) [noise=true , address=145.79.227.52:4447 , piece_id=O6YR5T34Z74FCZBRLJH73NGWB6MESJIS4R4LDHFCIXKNQHHGDE5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12fqT9xA7kXTG8K9XwTUxL6FXibV35AkPHDK6grKGHg17fk91fP, address:145.79.227.52:4447): protocol: rpc: dial tcp 145.79.227.52:4447: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830897168 (413h52m42.013342148s) [noise=true , address=145.79.227.50:4375 , piece_id=MI4HWY7YC5N3JV2NBTW235MF5ZEXL675RHVRL4JN55NBPTLYCBJA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:121fzX9v4rgEd78xnDAwcQtthyQHV3yUZUQYLXZ9byPDucGK4T5, address:145.79.227.50:4375): protocol: read tcp 5.161.89.252:59534->145.79.227.50:4375: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.830942051 (413h52m42.013387031s) [noise=true , address=145.79.227.34:4906 , piece_id=KLDGV4ZRNCWSMPXLFSV6VZOJMAEATT5SFDVTYKAKIUUFFHXOCVZA, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:12m9gfwSNSLJB2XyBAQUVtWUpc1jQtr5dvYWzHqXeaLQUzyW9cw, address:145.79.227.34:4906): protocol: rpc: dial tcp 145.79.227.34:4906: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831145001 (413h52m42.013589981s) [noise=true , address=145.79.227.48:4302 , piece_id=VQ5AJYOBHYKEOKRUZLXPP5U6ENN3CMOVUEJZTWD6UUFUY3ANMTVQ, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1tGpuYdPLiy4ncDjxrZbQfp7Bq79NVBHDmr4vjaL4qMpudrvgR, address:145.79.227.48:4302): protocol: read tcp 5.161.89.252:36936->145.79.227.48:4302: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831152034 (413h52m42.013597014s) [noise=true , address=145.79.227.40:4049 , piece_id=FFW5H3DDALNRLX4L7VLSF4RLXH6OGAY5NCUCBPJ27IVJSXUB2H2Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J, address:145.79.227.40:4049): protocol: read tcp 5.161.89.252:55322->145.79.227.40:4049: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Feb 1 21:49:03.831343792 (413h52m42.013788772s) [noise=true , address=145.79.227.47:4266 , piece_id=FCH75NBLOJFNOFUK75NVAO4I57GZ6C7F3TJ6JM5CQNRWI4OJCF5Q, seg_pos={0 0}, upload=363886]: Done uploading piece. err: ecclient: upload failed (node:1uzGTks5tffchQBGQK23fnWZXMfrsTHh4Zqs9UGwnWUmnQghEW, address:145.79.227.47:4266): protocol: read tcp 5.161.89.252:63120->145.79.227.47:4266: use of closed network connection; conn closed: op
Feb 1 21:49:03.831373287 (413h52m42.013818267s) [seg_pos={0 0}, upload=363886]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil>
Feb 1 21:49:03.846645708 (413h52m42.029090679s) [upload=363886]: Done uploading segments...
Feb 1 21:49:03.846661618 (413h52m42.029106598s) [upload=363886]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9)
Duration: 30.017278319s
core
uplink
us-select-1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [X...........................................................] duration:
271.469456ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:18:55 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 271.488591ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.X..........................................................] duration:
271.860415ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:18:15 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 271.898957ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..X.........................................................] duration:
745.247811ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:17:32 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 745.26445ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...X........................................................] duration:
276.265112ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:16:31 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 276.282395ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [....X.......................................................] duration:
278.212572ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:15:47 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 278.227871ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.....X......................................................] duration:
289.5129ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:14:56 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 289.52909ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [......X.....................................................] duration:
775.574914ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:14:08 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 775.592217ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
269.832138ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:13:09 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 269.850362ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
316.524851ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:12:20 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 316.56143ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.........X..................................................] duration:
815.989396ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:11:46 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 816.007871ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..........X.................................................] duration:
268.517927ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:10:57 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 268.539247ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...........X................................................] duration:
287.31289ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:10:25 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 287.32906ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [............X...............................................] duration:
279.174277ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:09:51 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 279.190917ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [.............X..............................................] duration:
266.226195ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:08:51 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 266.246794ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [..............X.............................................] duration:
301.827885ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:08:29 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 301.847773ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [...............X............................................] duration:
389.854203ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:07:50 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 389.868139ms
edge
linksharing
bgp
ap1
|
checkers.HTTP{"GET" "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt", ExpectedContent: "Hello, world!"} failure (antiblip window 6/60): (http.ap1.linksharing)antiblip:
1 out of 60 is failed (threshold=6) [................X...........................................] duration:
377.24134ms remote:
"tcp:136.0.77.4:443" target:
https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt trace:
https://jaeger.dev.storj.io/trace/6b1a2845b514c21
FAIL (2026-02-01 18:07:04 UTC)
orbiter warning: Get "https://link.storjshare.io/raw/jwlk53gfhsby563unjoqoivp56jq/uptime-checks/hello-world.txt": context deadline exceeded
Duration: 377.266287ms
edge
linksharing
bgp
ap1
|